If his garbage causes you take take a different flow of execution, however, that provides him a way to reach bugs in the little-used parts of your code.
The different flow of execution triggered by an overflow trap should almost always be a simple call to "abort()". At this point, your program has already failed and should be stopped.
I disagree with your premise. Garbage input values should be checked and rejected in software before the overflow ever occurs. The hardware overflow check should be a last resort to enforce this at every instruction step, and in the worst case it converts privilege exploits into less serious DOS attacks.
Allowing "garbage output" as you propose just creates more opportunities for attacks when that output gets consumed somewhere.