The session was the last chance for the Parliament to pass a long-delayed law in time for Bosnia to conduct a 2011 census alongside other countries in the Balkans.
The results of the census would help decide how many representatives from different ethnic groups would sit on lawmaking bodies, but ethnic and religious divides plague the Balkan country as it tries to attract foreign investment and make progress toward joining the EU.
The European Commission (EC) expressed concern over the failure to pass the bill.
"There is now a high risk that the country will fall much behind in its economic development because of lack of reliable statistical figures. Its European integration process will be further delayed," it said.
Bosnia conducted its last census in 1991 as part of Yugoslavia, before it suffered Europe's worst fighting since World War Two. It does not have reliable data on the population, half of whom were killed, driven out or fled during and after the war, or on its ethnic composition.
Following the 1992-95 war between Bosnian Serbs, Muslims and Croats, Bosnia was split into two autonomous regions, the Muslim-Croat federation and the Serb Republic.
Dusanka Majkic, a leader of deputies from the ruling Serb Alliance of Independent Social Democrats (SNSD) in Parliament's upper house, said they were boycotting the session to derail a law she said was against the Serb Republic's interests.
Majkic said if there was a vote, the SNSD would lose as two deputies representing other Serb parties in the upper house had said they would back the bill.
The law would allow the counting of all citizens who would state their ethnicity, religion and language.
It was approved by the lower house but the SNSD insisted on changing the provision saying that results of the census - such as deciding the ethnic make-up of lawmaking bodies - will apply only once most war refugees return to their Bosnian homes.
Serbs want the census results to be implemented immediately, while Muslims and Croats insist they should be implemented only once refugees return, to avoid cementing the mass exodus of Muslims and Croats in national institutions.
Serb Republic deputies often block state laws in the National Parliament to prevent the strengthening of the state. The Serb Republic passed its own census law in April.
Sulejman Tihic, a Muslim deputy in the upper house, said the move of the SNSD deputies violated the constitution.
"This is a dangerous precedent and it is happening for the first time that deputies of one party are blocking the Parliament because they might be outvoted," he said.
Failure to convene the session has also put on hold the approval of the state presidency decision to send a 45-man Bosnian army unit to join the NATO-led mission in Afghanistan in September, seen as a condition for NATO membership.
