I live in Finland. Why do you think there is "instability" in "having to form a coalition government"? To the contrary, I find it ensures that every vote counts to the extent it should. I also would be somewhat horrified if some party in my country got enough votes that it could form a government alone. It would have way too much power.
Let's consider a simple example, say we have parties A (40% of votes), B (35 %), C (15 %) and D (10 %).
To be stable a coalition should have around 60 % of members of parliament behind it. Hence, generally the interesting stable coalitions would be A+B, A+C+D and B+C+D.
Now this allows for the parties to haggle and trade on those issues they consider most important. For example, say that party B strongly wants to have higher taxes for fuel, while party A is strongly opposed to it. Party A somewhat prefers to impose fascism, while the other parties don't want that.
Now, say that parties C and D would, everything else being equal, not want higher taxes for fuel, but they don't consider it a core issue.
Now, let's say party C wants to safeguard some minority to a larger extent compared to the other parties, and none of the other parties is strongly opposed to that (but would not do it on their own). Party D has similarly some core issue of its own.
Now, A+B may not be a good coalition because their agendas are so contradictory that it's hard to figure out a good deal. A+C+D would be a viable coalition: A, C and D get their wish on fuel taxes, but A has to give up on fascism (which it mildly prefers) and accept the core agendas of C and D.
Also, B+C+D would be possible; probably B would have to make some concessions that A is not willing to make.
As you can see, every party has an incentive to offer to give up to an extent on their minor agendas in order to reach a consensus that is preferable to each of them. And this, I believe, makes everybody get a better deal.
Also, a two-party system (which is a natural result of your voting system) seems to be incredibly polarizing. See, you have two kinds of people, democrats and republicans, and it seems to me that the ones you don't belong to are those crazy lunatic bastards. I find it hard to imagine that is healthy for a population.