Sure, self-driving cars will frustrate us in new ways and it's always great to address potential issues prospectively.
However, just because you are used to all the frustrations and inconveniences of the current system and you're just thinking about the annoyances of self-driving cars for the first time doesn't mean the new annoyances are worse than the old.
Those of us who live on obscure streets or on divided streets have had to talk taxi drivers, friends and delivery people to our houses for years. GPS has made much of this easier but the whole complaint here is that google doesn't know exactly where every address is.
Yes, self-driving cars remove the option of talking the driver in despite the GPS error. But in the long run we are better off if we are forced to learn once that "Address X" will bring any visitors to our home rather than having to give them all directions. Not only is this an easy simple fix it actually forces us to do what we should have done a long time ago.