I took an english class in college on the "rhetoric of intimidation". that is, how to write in order to intimidate. Not surprisingly, example #1 was the IRS. the professor had spent years studying with them and working with them. One of her favorite stories was when she learned that the majority of the time, an audit occurred because the IRS's records on you didn't match, and rather than figure it out they audit you to make you figure it out.
My best friend works with a lot of self-employed people. One of them had several (3 or 4) years in a row where his tax refund would have been miniscule, something like $10 or $20 in the black, so he didn't even bother sending in the forms. He figured he'd just let the government keep the money. The IRS responded by sending him a bill for roughly $10,000 owed, because they figured that was a nice round number to make up.
I'm sure everybody here has anecdotes like this. That "1%" of bad eggs you talk about must have been terribly terribly busy.