Amen, brother! I prefer to get my bills in the mail. Real, honest to goodness paper.
On the outside of the envelope I write the date I received the bill, how much it is, and when it is due. It goes into my mail sorter in the first bin. When I pay it online I write the confirmation number from the bank on the envelope and then put the envelope in the second bin. I keep the envelope until the next bill from that company comes in. That way I can see if the previous bill was actually paid on time or if there were any mistakes on their part or mine. If everything is okay, I shred the old bill, envelope and all. Then the cycle starts over again.
Important items like house payments or car payments are kept until they're paid off. I want a physical record I can use to prove I've made all the payments and that they were made on time. No, I don't trust banks or mortgage companies any further than I can throw them. Do you?