"Completely unsafe?" How many humans drive every day with no incident vs. how many accidents occur? I drive to work every day and don't wind up crashing into a car each time. In my entire driving career (about 20 years), I've been in 4 accidents. (Only 1 of those my fault - though the insurance company disputed the fault of a second one.) I couldn't tell you how many miles I've logged that resulted in me getting to my destination without any harm to me or my passengers. Going by days driven vs. accidents, though, I have about a 0.05% chance of getting into an accident when I set out on the road. Put another way, I'm a 99.95% safe driver.
What's the track record for a theoretical Consumer GoogleCar? It hasn't been released yet, so it's completely unproven. It might be 100% safe. It might only be 75% safe. If it is 90% safe, it will be less safe than me. If it is 99.99% safe, it will be a safer driver than me. The point is that I don't know. Why should I put complete trust in something, eliminating any backup system, when that thing has no track record? Because the company assures me it is safe?
Keep the manual controls so people have that manual backup. If the automated cars are as good as they say, people won't use the steering wheels and the need for them will go away. However, ripping out all backup systems and putting your trust in something with little to no real world experience is short sighted. I'm all for embracing new technology, but bleeding edge shouldn't be referring to possible automated car glitches.