My family owns a business in light manufacturing. When one of the workers gets a new tool set, they literally throw away the metric tools, because they're completely worthless in typical industrial use. Multiply that imperial-unit inertia by about a million small hardware-related businesses and manufacturers across the country, and you can see why no one has been eager to swallow the cost of that conversion.
I agree with you about the tooling and infrastructure issue. Go to a local home depot and check out the selection of nuts, bolts, washers, screws, pins, connectors, etc. Most of those are still in imperial units. Some people "pooh, pooh" the actual cost of a real, complete conversion from one unit system to another. Most of those people tend to neglect the actual hardware in use today, and how pervasive those units are throughout the entire US manufacturing base. It would literally cost billions of dollars in conversion costs, and in the end, all we'll end up with is a more "mathematically pure" measurement system - zero functional difference. To local businesses and firms, there is literally no benefit to the conversion in the short term - only cost. The government would literally have to mandate a change by law to kickstart this, and it would be a short-term but real hit on the US economy.
It would be great if the US could switch to metrics. Most people - educated ones at least - understand it's a saner system and is better both for internal consumption and for international interoperability. But it's not what we have, and no one currently believes the conversion is actually worth the price we'd pay. It's one of the prices the US pays for having such a large, isolated infrastructure. It's harder for us to adapt to changes in some ways because of our sheer size, and there's much less pressure externally than with smaller, individual markets. The cultural resistance is not insignificant either, but that could be overcome with time.
Probably the best way to make it happen is for the government to provide some tax benefits to companies willing to do the conversion, and allow the transition to occur a bit more naturally over time. That would help to disguise the cost (there's no free lunch there, though), and eliminate some of the grumbling, and as such, some of the political opposition. It's not enough to just label things in different but equivalent units. Until you make the internal conversions in the low-level infrastructure, all you're doing is creating more work by superficially labeling things less efficiently.