there are on-road and off-road drives that are a lot of fun for the enthusiast and many such as I won't want to give up the option for that.
While I'm sorry about the loss of a fun activity, we don't spend tens of billions of dollars per year on roads for entertainment. If the best way to make the roads efficient and safe would be to get the human drivers off of them, I'd vote for that in a heartbeat. (I don't know if that is really required or not, but I suspect that roadways could be a lot more efficient if we eliminated the human factor: closer following distances, no need for yellow cycles on traffic lights, better planning for capacity, etc.)
People can use the roads for leisure driving now, because every other driver on the road is human and we have built everything to take that into account. But I just don't see giving up potentially billions of dollars in efficiency gains to preserve your ability to have fun on the roads.
There's probably some compromise that sets aside some roads for you, and maybe even sharing the roads at certain times when there are few other drivers. But I think a lot of people, including me, will consider that a lower priority.