The problem is that you confuse manual controls and manual control inputs. What type of control a person or computer uses is irrelevant. What is relevant is the mind making the decisions about what inputs to make. Computers are yet to be sophisticated to handle many situations as well has humans do.
Some humans. A huge proportion of mankind don't know how to drive. I don't. Most of my friends don't. Kids and handicaped people can't, legally speaking.
I'd rather have automated cars that can move untrained people, children and handicaped around, than have error-prone stragers do it (eg: buses).
Would you rather trust a stranger than a machine?
And then, let's not forget the traffic factor. I live in an area of my city (Buenos Aires), where it's faster to walk 20 blocks than take a bus or cab (about twice as fast, btw). On one hand, because there's a stupid amount of peolpe moving around here, but on second hand, it's because they're awfully disorganized and inefficient. They each want the shortest route, and end up clogging everything instead of coordinating for greater overall efficiency. Humans simply can't do that. And generally, wouldn't if they could either.