SF has always been a bedroom community for Silicon Valley, to some extent: I moved to SF from the Peninsula in '93.
Caltrain is simply not capable of being a full solution to the problem of getting people out of their cars, so the buses are a very reasonable solution. I'm lucky enough that my company moved to SF this year, so it's Muni every day for me. (Sometimes a mixed blessing!)
Not to be too cynical, but San Franciscans will always have something to complain about. My grandmother didn't like all the "new development" out in the Sunset, which since it happened in the 1920s gives you some perspective. I love my crowded quirky little city, and certainly don't begrudge the Apple, Google, and Genentech buses in my neighborhood, though I do wish they'd coexist better with Muni.
I sort of laugh at the people being shocked at "houses built right next to each other": um, have you never been to a city before? (I'm talking about Paris, Manhattan, London, or Tokyo, not someplace spread out like Phoenix.) We have the density but not the height, lots of trees and parks, and many neighborhoods are very walkable.
In short: cities are dense, people like to complain, and private mass transit is good at moving people from once place to another.