Also, considering something like Star Trek, there is some comfort in imagining a society where intellect and honesty are rewarded rather than ridiculed. Which, of course, can be a central aspect of speculative fiction; how far can an ideal geek society go, or do you need someone to be an asshole to take action in the end.
However, I agree with the OP to some extent. As a science/math/electronics/programming/music geek, I've never understood the stereotypical geek fascination with games, comics and other plasticky entertainment. Reading books I can understand, collecting superhero figures not that much.