You can't eliminate the Irs even in a flat tax. Doing so means you stop caring whether or not people are being accurate with their tax bills.
99% of the irs job is verifying that the income you report and pay taxes on is equal to the income you actually took in.
In a country without an irs net revuene from taxes would fall to a million dollars after the first year or two. I know I would only claim an income of a few hundred dollars. As I would count everything I spent first.
Stop being stupid.