It's better to let a guilty man go free then to put an innocent man behind bars.
That may have a shred of truth, but it's not infinitely better.
One criminal not behind bars can kill 1000 or steal from thousands. One innocent person behind bars is 1 life diminished (but not extinguished).
And police can know for a fact that at least one of those two person's is guilty of something. So if the defendant is deemed guilty, among the suspects available to hold guilty of a crime, there are very very good odds.
The courts only have a certain success rate, and the number of people arrested in the US is large -- more than most other countries.
Therefore, the law of big numbers tells us essentially MANY innocent people will be erroneously placed behind bars for long periods of time, regardless of what we do, AND MANY guilty people will be erroneously allowed to go free, regardless of what we do. These are facts of life, which cannot be avoided, only accepted.
Both errors are grave. Arguably, having more criminals go free, increases the rate of crime on the streets.