I listened to an economics podcast that discussed the role of automation in the economy.
The Google car came up and the MIT professor who was being interviewed said that contrary to what a lot of people think, the Google car is highly dependent on very detailed annotated maps and can't just detect stuff like traffic controls (lights, signals, etc).
I can see a driverless cars happening in urban areas if "self-drive" features start getting built into roads, like RFID chips embedded in lane stripes, traffic controls that have RF signalling. Even then you would probably need car-car signalling.
But at some point it seems like they aren't driverless cars as much as they are trackless trains or some kind of personalized mass transit. Decide you want a ride someplace, request a car and it follows GPS + signalling to your house from its parking place and then delivers you to a destination. Uber without the driver.