That's because you can be unsafe and not crash. You are defining "unsafe" as someone who has crashed. Not as someone driving poorly and unsafely.
No, to me "unsafe" means elevated risk of harm from an activity beyond the expectations for that activity. So one important way to study what is unsafe is to look at what sort of groups or behaviors are involved in harmful consequences.
You have very high expectations for risk from automobile accidents, but why should the rest of us share your expectations?
I'll present a US-centric case and examine mortality rates (since those are well documented). If the only way we could die was by accidents or injuries (including suicides and homicides), then we would have an average life expectancy (once you get past the dangerous years of childhood) of almost 1700 years (due to a 60 deaths per 100,000 people in the US). That increases to roughly 2500 years, if we exclude intentional causes of death (40 deaths per 100,000 people). Of this, motor vehicle deaths make up 10.8 deaths per 100,000 people. So if we could eliminate that as a cause of death we'd increase human life expectancy by a considerable amount 300 years for the former case and 800 for the latter.
But we don't live in that sort of world where it makes sense to go hardcore on reducing highway deaths. Instead we live in a world where the US has a death rate of 800 per 100,000 people and even complete elimination of highway deaths won't have much effect on our lifespan since most deaths are due to illnesses that come upon us when we get older.
Second, there is this unwarranted assertion that we can make self-driving vehicles substantially safer than any human driver. You present no argument for this other than to assert that the best of human drivers are "unsafe". However, if that were true, then you would expect that the most unsafe drivers, the drunk drivers and those who who can't maintain a valid driver's license would have a far smaller share of the highway deaths than they actually do. Everyone should be contributing significantly, not just the very worst.