The government brought Ahlu Sunna Waljamaca on board ahead of an expected military push against hard-line Islamist rebels threatening to topple the administration.
"Together, we are going to eliminate radical Islamists from the country. We will confront Shabaab directly not through the media," chairman Maalim Muhamud told Reuters late on Saturday.
Muhamud said his group, which controls large swathes of central Somalia, had the capacity with the government, to ruin Al-Shabaab, which professes loyalty to Al-Qaeda and holds vast areas in the south and the capital.
In January this year, Al-Shabaab attacked Ahlu Sunna's positions in a bid to take control of strategic towns, but the Sufis defended them successfully.
Under the deal signed between the group and the government of President Sheikh Sharif Ahmed, Ahlu Sunna will get five ministerial posts and appoint the army's deputy chief of staff.
The Sufis' quarrel with the rebels is mainly ideological.
Somalia has a rich Sufi tradition going back more than five centuries. Sufis have been angered by the desecration of graves, the beheading of clerics, and bans on celebrating the birth of the Prophet imposed by the militants.
The latest round of grave attacks occurred this week in Mogadishu after similar incidents in Kismayu and Baidoa in the south and in other areas, over the last two years.
"This is an unacceptable matter. The ones who are doing this are not true Muslims, they are far from the religion. We must launch a jihad against them," Muhamud said.
An Al-Shabaab official said the alliance between the Sufis and government would not impact the balance of power.
"We have heard from the media about the deal they signed in Addis Ababa, but it will not have any impact on us. Our Mujahedeen are ready and are well trained," Sheikh Ali Hussein, chairman of Al-Shabaab in the capital told reporters.
The deal between Ahlu Sunna and the government was opposed by several members of the militia group, including Muhamud's deputy, Hassan Qorey, who says they were not well represented in talks that led to the agreement.
"Yes there are some Ahlu Sunna Waljamaca scholars who are opposed to the deal we have signed but we are going to solve our problems through talking to the rest of our group," Muhamud said. "On the government side there is also some opposition, so we hope the other side will also do so."
The chairman would travel to Mogadishu this week for a meeting of a joint technical committee of the alliance on implementing the power sharing deal, he said.
Somalia has had no effective central government for 19 years and Western nations and neighbors say the country is used as a shelter by militants intent on launching attacks in the region and further afield.
The Islamists launched their insurgency at the start of 2007 to drive out Ethiopian troops propping up the government.
Ethiopians left at the start of 2009 but the conflict continued.
A smaller rebel group, Hizbul Islam, has an alliance with the Al-Shabaab in Mogadishu, where the government has been hemmed into a few blocks since a rebel offensive last May.
