At least for some of us, it's not just basic arithemetic but having to spend tons of hours tracking down information that *could* be readily provided but, for some bizzare reason isn't. For example, have you ever had to deal with a ESPP in a company? Here's an example:
Fidelity will try to keep track of your wash sales and adjust the basis of your future sales accordingly. Unfortunately, that makes the record-keeping almost impossible. Firstly, adjusting the BASIS doesn't just change the basis price, it also changes the "effective" DATE of the purchase. So when you go to sell that March 31st stock, may get statements from Fidelity showing that you sold stock bought on "12/31/2009w" -- it's really your 3/31 stock but the basis has been adjusted due to your wash sale on April 5th. This can get extraordinarily confusing, because it means that neither the BASIS price nor purchase DATE for this set of shares corresponds to what was actually sold.
And that's a freakin' disaster, because as mentioned 3 sections ago, Fidelity fails to properly adjust the BASIS price of shares ESPP sales, meaning that you have to do it yourself. But since Fidelity has already "helpfully" changed the BASIS price and DATE to account for the WASH SALE, it can be virtually impossible to figure out WHICH shares were actually sold, meaning it's virtually impossible for you to properly adjust the BASIS to take into account the ORDINARY INCOME you recognized.
Oh, and keep in mind that the change of the BASIS DATE means that your later sale may be changed from a SHORT-TERM transaction to a LONG-TERM transaction. Confusing.
Oh, and keep in mind that because you often will be buying and selling different-sized blocks of shares, a given sale will likely be split up into multiple transactions, each tracking a different wash sale.
From: http://www.ericlawrence.com/eric/show.asp?entry=4/2/2010%2012:20:00%20PM
I agree that a basic 1040 is easy, but there are tons of gotchas out there due to not only the legal complexity but also how your company and/or broker does their bookkeeping. Reverse solving to uncover lost data is a huge pain, especially if you're like some of us who opt-out of paper statements. Some of the critical information you need to know is just gone.
Of course, if you're one of those people who keeps a dozen filing cabinets and spends 30 minutes a day doing personal finance and bookkeeping, sure it's trivial. Most people aren't like that and shouldn't be required to be like that.