And, yet, you should be careful about which academics you listen to, as well. Many folks go straight from undergraduate work to graduate work to a post-doc, and so on, without ever experiencing industry (and if you're gonna protest, "What about that summer I spent working for IBM?" then you're part of the problem) or, often, any part of the world of work as the rest of us understand it. (And, again, "I worked in the library shelving books for work-study," ain't it.)
Given all that, "Gee, the work's not so hard and the rewards aren't so slight," might not be very accurate. Asking the rest of us, who've actually worked in the "real world" (for some definition of "real world") and also watched academics in action from close range (parents and partner and working at research institutions), might be useful.
It's possible to find a rewarding (and not just financially rewarding) job that let's you use your skills and brainpower to change the world for the better. I'd argue that it's not even that hard to find one, depending on your skills and talents. If you want to get a PhD, more power to you, but make sure that you understand the tradeoffs you'll be making.
If you do go for it, go in with the understanding that academia as we know it is facing a bunch of significant challenges -- funding, distance learning, etc. -- and it may go through some pretty radical changes in your lifetime.