If you assume an ex-prisoner will commit crimes again, your prison system isn't working.
At least in the U.S., it's a good bet that a criminal will re-commit. This may be a sign that the prison system isn't working, but it doesn't change the fact that we have a recidivism rate of over 50% in the first year after release alone.
That said, if we don't give "rehabilitated" convicts the benefit of the doubt after "paying their debt," we're pretty much guaranteeing that they'll have to return to crime. Convicts do need the ability to escape their criminal past.