What's dangerous is 3,000 pounds of metal being controlled by a driver who is impaired by alcohol, drugs or messing around on their phone. Around here the greatest impairment is age. A good third of the people on the road around here can barely see. Self-driving cars don't have achieve some lofty safety record to become the standard, they only have be better than humans and that's already within easy reach compared to the technical hurdles already overcome.
4. No one controls when and where I go
That may be the dumbest excuse to oppose technology I've ever read. If you fly, ride the bus, train or cruise ship, other people control where you go.
I remember people in a video forum in 2004 telling me they'd be shooting film the rest of their lives. That was just 11 years ago. In just that short time span video has not only rivaled film but surpassed it. Long before video surpassed film in terms of quality, video displaced film on the basis of cost and ease of workflow. The technical hurdles in 2004 for video to replace film were huge and it happened in less than a decade.
Cars are not only going to rival human drivers but surpass them, and definitely a lot sooner than you think. It won't be that long before people who insist on driving themselves become the hazards on the road and I don't think your right to seize the steering wheel is going to trump the lives of other drivers.