Sogavare lost the vote 27 to 23. The vote on Monday in Honiara was triggered by a series of defections from Sogavare's coalition government late last month, Radio New Zealand International reported.
The vote followed an arduous debate that stretched over six hours. Supporters of the motion said MPs had lost confidence in the prime minister because he was not listening to them anymore, while development was stalling around the country.
Sogavare's supporters urged MPs to let him finish the final 10 months of his four-year term in office. But the East Honiara MP Douglas Ete said time had run out for Mr Sogavare in his third stint as prime minister.
"Ten months is too much time, Mr Speaker, enough time to destroy this nation."
Meanwhile, the parliament is expected to convene later this month to elect a new prime minister. Until then, Sogavare is the acting prime minister.
The opposition grouping has not declared a candidate yet; it is understood that there are several MPs preparing to put their names forward.