Officials and witnesses said interim government forces took back the administration headquarters in Osh and Jalalabad a day after backers of Kurmanbek Bakiyev seized the buildings in a challenge to the new government's authority.
The showdown has underscored worries about stability in the mainly Muslim, ethnically divided nation which hosts US and Russian military air bases.
Bakiyev, who himself came to power as a result of a revolt in 2005, fled the Central Asian nation after an April 7-8 uprising, and has taken refuge in Belarus.
The United States and Russia have expressed support for the interim government and are keen to prevent any further unrest in the ex-Soviet republic strategically located near Afghanistan.
On Thursday, Bakiyev supporters seized government headquarters in all three southern provinces — Osh, Jalalabad and Batken — and also took over the Osh airport forcing Defense Minister Ismail Isakov to arrive by helicopter.
Interim authorities said they regained control in remote Batken late on Thursday.
On Friday, a crowd of their supporters marched to the government headquarters in Osh, the main southern city, and took it over after scuffling with guards. No serious injuries were reported.
In Jalalabad, a day-long confrontation that left two people dead and dozens wounded ended when interim government supporters took control of the administration building, officials said.
"A special operation is under way to comb Jalalabad's area and catch the ones who were shooting," Temir Sariyev, a deputy interim government head, told Reuters. "Bakiyev has now shown to the whole world that he will stop at nothing."
Sariyev said units of vigilantes had been set up across the nation "to disrupt any attempt to destabilise the situation."
The European Union earlier called on all "political actors" in Kyrgyzstan to refrain from violence and finalize the draft constitution, bring it to popular vote in a referendum and hold fair and transparent elections.
