It's not like convicts get out of prison and they're reset to a neutral state and can try hard and do ok in life.
Ex-convicts are actively persecuted by society. It'd be like if you fell off the wagon and then a buffalo decided to sit on you.
It's not just "on you" to get back on the wagon.
And a significant portion of the population is now an ex-prisoner or ex-felon. "In 2008, about one in 33 working-age adults was an ex-prisoner, and about one in 15 working-age adults was an ex-felon. Among working-age men in that same year, about one in 17 was an ex-prisoner and one in eight was an ex-felon." http://www.cepr.net/press-cent...
Millions of people. Your short sighted "I personally don't care about your well being because you fucked up and I'm scared of you" mentality would be like saying, "Why should I pay taxes for public schools if I don't have kids?"