Self-driving cars should be the legal equivalent to sitting in the back of a taxi. Even from an insurance/liability standpoint, owning one means you're responsible/liable for fuel & maintenance - and that's about it. It should be down to the manufacturer to ensure safe, autonomous operation.
The autonomous car is safe only within its operational limits --- but how many drivers will be willing to let a car or its manufacturer decide when it is safe to take to the roads?
How many will risk being stranded if automated systems begin shutting down because they are confused and overwhelmed by bad weather, outdated maps, or other unforeseen circumstances?
How many drivers? Probably not that many, unless the safety envelope is very wide indeed. However, if the autonomous car is, in fact, autonomous, the question is also how many non-drivers would like to have access to the road, most of the time, without becoming drivers. Especially if they don't have a choice(visual or other incapacity that precludes driving, alcohol violation, too young, etc.) or their use case is relatively miserable driving(If you are going to have a shit commute in heavy traffic, do you want to 'enjoy' the 'freedom' of being the master of your vehicle, or spend the time doing something that sucks less?), the value of becoming a driver may not exceed the hassle.
The initial systems are going to fail miserably in any environment less constrained than a test track, so they'll be too limited to be of much use to anyone who can't also drive them when the need arises; and there is a strong cultural value associated with gaining access to driving capabilities(in some suburban areas, you basically aren't a real person until you can drive; because if you can't do that you are homebound and dependent); so I suspect that swaying existing drivers will be harder; but convincing future non-drivers to just never bother to head down to the DMV at all, and instead employ an autonomous vehicle either casually(like zipcar; but with autonomy) or for more lasting lease or ownership arrangements might well be easier.
People already put up with stranding risks of various sorts(it's not as though cars breaking down is a thing that requires computers, and mass transit, cabs, car-pools, etc. have their own failure modes), so as long as they skew more toward 'annoying' than 'overtly lethal much of the time', people will definitely risk it if the convenience is sufficient.