As someone who has been developing custom CMSes at various jobs for over a decade, I've gotten to know what makes a CMS work (or not) quite well, and I've given most of the popular PHP-based open source CMSes (especially Drupal, Joomla, and MODx) more than a fair chance to impress me.
So far, I always eventually get frustrated with their arcane and idiosyncratic ways of doing things, which often require as much new specialized knowledge as learning a programming language in itself, and in the end they're never quite the right tool for the job anyway.
Now that I've been freelancing full-time for a year, I've come to a decision I'm satisfied with: as far as I'm concerned, there's only one pre-built CMS I've never been disappointed by, and believe it or not it's WordPress. But WordPress of course is not ideal for every site (it IS, however, ideal for lots of small- to medium-sized sites, not just blogs). Concurrently I've been developing my own custom CMS on CakePHP, tailored to the specific needs of my clients, and expanded as necessary as I've been adding more clients who are using it.
So, what it boils down to for me is this: assess the client's requirements, determine whether or not WordPress fits the bill, and either go with WP or use my own proprietary CakePHP-based CMS, expanding it as needed.
Maybe if one of the "big" open source CMSes were built on one of the equally "big" open source MVC frameworks, it would be worth investing the effort into getting to know inside and out, but just as I've made the decision to focus on PHP as my development language of choice, I've also had to decide to focus on one framework (CakePHP) and one CMS (my own) to know intimately and be able to build upon as needed.
It's not so much a matter of "staking my career" on it as it is of focusing on my strengths and building upon them. The whole point of freelancing is to get more bang for my buck in terms of productivity (and income) per hour worked. And so far, Drupal, Joomla, MODx, whatever... they just don't fit into that model for me.