The more people you have, the more hardware manufacturers and software companies take notice and actually care about Linux. When hardware companies release open source drivers (and even proprietary drivers - look at nVidia) and when software companies release software for Linux (Wolfram's Mathematica, MathWorks' Matlab, National Instruments' LabVIEW), Linux users do benefit. As more applications become more available on Linux, it becomes a more mainstream desktop OS and the interest in Linux grows and quality of software for Linux improves. Once Linux is mainstream, more and more hardware will "just work" because the Linux kernel includes more drivers. The more the hardware "just works", the easier it is to configure for everybody using clickable GUI tools instead of looking up commands and command options (the command line, while sometimes easier to use than GUIS, is not necessarily hard to use for everything, but it is hard to initially learn). The easier it is to configure, the more "Regular Joe" users will be able to use Linux, not only making Linux a more visible solution to Windows, but making all of FOSS a more visible solution to proprietary software. Finally, as the number of Linux users increases, presumably the number non-duplicate bugs are reported (as some users are technically inclined enough to submit a bug, but not enough to fix it - something even Linus, if I remember correctly, has noted as an advantage of FOSS), making architecture errors easier more visible, giving developers to improve the architecture, which hopefully leads to happier developers and users. All of these changes, while insignificant when standing alone, lead towards a better world of software for everybody, including you.
So whether or not Linux developers are aiming for world domination or not, the "Regular Joe" users of the Linux desktop are as important as the developers themselves.