While the libertarian in me agrees with you, the realist in me sees some problems with the argument. The problem with letting those who made the choices pay the piper is that usually, I (taxpayer me) still end up paying, and usually much much more, in the long run. The person who had kids who couldn't afford them usually cannot care well for them, which tends to lead to disadvantaged kids with no/minimal education, which leads to higher crime, more poverty, and a higher burden in the future.
The libertarian in me despises the idea of nationalizing most of health care, but I'll end up paying more if everyone without a job goes to the emergency room and I pay (much) more because of it.
If we can't find a way to help people afford houses, then they lose the houses, and I end up paying more property taxes to support the streets, water, and other things I like to have. And then we get Detroit.
In the end, I've decided that the "best thing for me" is to help other people screw me less... which means figuring out how to help them help themselves, not leaving them until they're a burden. I disagree with many of the tactics we use to achieve it, but leaving everyone who made a bad choice to flounder in the sewer costs me more in the long run.