I think we're always going to need to rely on a human operator fallback as a fail-safe and we'll never be able to stop enthusiasts from manually operating their vehicles. If that's true we'll still need to be licensed, insured, sober, and mostly paying attention.
There are alternative solutions to human operator fallback issue. The car manufacturer or software group could have a central hub where the cars ping a human overseeing multiple cars for the appropriate action to take. To deal with mixed autonomous and manually operated vehicles we could have designated roads / lanes for such activities.