What you cannot see, is that prisoners are bad because they murder/brutalize innocent people,
You have erroneously conflated "prisoners" with "violent offenders". The vast majority of people convicted to custodial sentences are convicted of non-violent crimes, such as possession of illicit substances or petty theft. Even car-jacking is usually carried out in the absence of the owner and with no threat of physical harm.
It is not only the violent criminals that are alienated and disenfranchised by the "ex-con" label, but anyone serving a custodial sentence.
And even violent offenders may be victims of circumstance. Drug addiction doesn't come without the addict choosing to take drugs, but the consequences of drug addiction can include violent tendencies that are cured if the addict cleans up. But what's the point coming off your escape from reality when reality is that no-one will give you a second chance?