Humans are terrible drivers. We drive emotionally, we get fatigued, we get bored, we drive too fast, we zone out, we fall asleep at the wheel, we have wierd medical mishaps like heart attacks and epilepsy and fainting spells. To top it all of we voluntarily impair ourselves with alcohol, drugs, text messages, and staring at ads/women/wrecks/houses/scenery.
How old will you be in fifty years? As you get old and decline in skill and health, do you truly think there won't come a point where it's smarter to trust your life to the algorithms instead of your own failing mind and body?
Self-driving cars will be a blessing for humankind. Not only will they save you from yourself, they'll also save you from all those other dumbasses out on the road. They'll give the elderly their freedom back, and open up new transportation options for children, the blind, the inebriated. They'll likely transform cities in ways we can't even imagine: probably some balance between easing congestion and allowing parking lots to be re-purposed.
But we have got to go thru an intermediary period first. Tesla was the first manufacturer with the gumption to release such a feature; others will eventually follow. As with commercial aviation, each death will be learned from and used to make the systems progressively better... it won't be a big wasted opportunity like the ~30,000 fatalities/year we currently have in the U.S.