I agree, speed is important, but the flexibility of Gentoo is amazing.
I tend to refer to Gentoo as a tweaker's dream.
I came from the days before we had useful package management systems in Linux... back when you had to compile all of your packages and dependencies yourself. (X and a window manager sure was a lot of fun to install back then!)
So I remember the time of compiling everything and getting everything properly optimized for your hardware (and other software)... so for me Gentoo is kind of the best of both worlds, package management, but still doing it from source.