'Unfortunately', we also have a responsibility to ensure the safety of others, not just ourselves. When you drive your car, it's not just your life on the line.
For that very reason, as soon as self-driving cars reach a critical point of safety over normal drivers (perhaps at least 25% lives saved?), it's a GOOD idea to implement it ASAP. I like controlling cars too, but the roads are not for joyrides.