There are an incredible number of obstacles that a person can instantly recognize that even today, a computer can't.
And that list gets smaller and smaller all the time. When you look at the progress that's already been made, the last mile of automation is totally achievable. Around here a good third of the drivers on the road are old people who can barely see past the hood and another 10 percent are functional alcoholics. The handful of relatively competent people are texting or talking on the phone while they're driving. As someone who commutes by bike in this metallic soup of human incompetence, the transition to self-driving cars can't happen fast enough. I'll take my chances with machine intelligence and machine reflexes any day.
The underlying assumption to your comment is that human drivers are competent. That may be true in BF Nowhere Utah but, here in the civilized world, human drivers are fucking idiots. So, sure, go ahead and drive your pickup to the market. Around here I'd rather see a computer at the wheel than grandma.