The car is either autonomous, or it isn't. If it isn't autonomous, I'll drive it myself and be in control the whole time.
Let's try it on an airliner: "Either the airliner is autnomous, or it isn't. If it isn't autonomous I'll drive it myself and be in control the whole time."
Doesn't really work out. The pilot-in-command is always responsible for the safe operation of the aircraft, even when the aircraft is "flying itself". Now of course there are limits to this responsibility, if an engine falls off due to shoddy maintenance that's usually not considered a pilot error. But complex computer malfunctions are more often than not blamed on the pilot (too much so to my mind, but that's another question).
I foresee that we'll see the same development with "autonomous" cars (i.e. cars with advanced auto pilots). They'll do better on average than a human, but when the malfunction you (the "driver in command") will be left to pick up the pieces, if there are any pieces left to be picked up. As with piloting, you're average workload will decrease substantially, but when things to wrong, you now have a much more complex situation to deal with, and no time to do it. The maximum requirements on your performance actually increased even though the average decreased. If the NTSB can still say "pilot error" 99% of the time, then it'll be "driver error" 99% of the time with autonomous cars.
And like with aircraft you'll like it, since the average is long and dreary and malfunctions will be so few and far between that you can functionally ignore them.
"Sleeping in the back seat" will be just as much frowned upon as it would be in an airliner. Leaning back and having a cup of coffee would be about as far as you could stretch it.