I'd say that it's mostly disillusionment. You're right: it's not that hard in essence, but in the modern age of spam the effort required to stay clean is (IMO) non-trivial, and things can get very ugly very quickly. (Ugly in terms of bounce loops and bounce spam and various spools filling up.)
I gave up, but haven't switched to gmail (yet). I just turned off my external SMTP service, and now configure all of my MUAs to SMTP directly to the upstream server of the account I'm responding as. Most can handle that, and it still lets me run a unified IMAP server, which I love, but avoids all of the pain of black-lists and most of the pain of spam. Still run a local spool, because my ISP's mail is only POP, so I fetchmail from there.
So: the OP can probably get most of what he wants with an IMAP server (dovecot) and a web-mail front-end (maybe roundcube? I just read about it here.) Let the big ISP's mail spool handle MX and most of the spam filtering...