In SF there's scooters on every corner just waiting to be used... and most aren't.
So in the driverless model, to make it so i'm not wasting time waiting for a car to take me to the store, and then another one to pick me up and take me home. Is someone going to foot the bill to have thousands of cars just sitting around waiting for someone to click "i need a ride" button on their phone?
Who cleans up the driverless car if the previous rider gets sick in it or spills their drink? If the car shows up and their a slurpee spilled on the seat, I now have to reject it and wait another X minutes for another car to show up. Not exactly something I'm willing to do if I need to get somewhere. Also, how do you budget your time when you need to take into account the variable availability of one of these cars?
I can't imagine trying to haul kids around in these things in the case when you've got a child seat. Or, you want to take your bike somewhere, and you've got to attach a bike rack to it.