If it runs through my yard on the way to your location, then there's going to be a problem, a BIG problem.
You see, all maps have errors - printed maps, Garmin maps, Google maps, ALL maps. Garmin shows streets that aren't there and misses streets that are there. Google maps shows my house two doors down from my house. I've been using maps on my job for the past forty years. I've seen it all.
Also, street names/numbers are not permanent, they change, and so do addresses. New streets and neighborhoods are built all the time. Hell, I've been on streets where odd and even are on the same side, and where houses right across the street from each other have the same number. I've seen houses on streets where the house numbers run like - 402 next door to 1607 next door to 723. I've seen street signs with four different names for the two streets that intersect. BTW, it's not the houses that are numbered, it's the lots. Even vacant lots have numbers.
Mapmakers insert some errors on purpose. They can tell by the errors if somebody else is using their data. But even traditional paper maps have errors. I bought a map years ago (Rand McNally) that switched the names of two cities. How can you trust a map for tiny details when they can't even get the names of major cities right?
Humans can adapt to changes quicker than machines. Maybe one day AI will be good enough to totally trust. We're just not there yet despite all the wishful thinking.