"There may be times when they want the cars to drive them, but they won't be buying autonomous-only cars."
A future where people can opt out of buying an autonomous car sounds great but it's not feasible economically. Traffic lights and traffic signs are all things needed for cars being piloted by humans, autonomous cars don't need them. At some point we'll be spending billions maintaining human-readable infrastructure and road rules when there are fewer and fewer actual humans driving.
It's just like the Sunpass you use out on the tollway in Florida. There are fewer and fewer options for driving on the tollway when you don't have a sticker. It won't be long before it's mandatory. It's the same with autonomous cars. Once cars start to take over the day will dawn when we don't want to collectively maintain the signage, traffic lights and human readable infrastructure.