There seem to be two main arguments, all of which (I'm reliably informed by my parents) were argued to death in Australia before our successful metrification starting in the 1970s.
1. Imperial is entrenched.
It was in Australia as well; alas we bear the same burden you do with our British colonial heritage :). All our road signs, car dashboards, units for commerce etc were Imperial. We stuck things over signs, children were taught both systems, commerce migrated. With packaging in both units, the US is halfway there.
2. Imperial units make more sense.
I can't speak personally for this, because I grew up using Metric. For the arguments that Celcius is less granular than Fahrenheit though, may I introduce the decimal point (that Americans so famously applied to their Metric currency while the Brits were still arguing over shillings). Most of these arguments appeal to familiarity, which are valid for the time, but will fade.
I'd say 2.1: All units are arbitrary. Indeed, all the more reason we all use the same ones, rather than having two systems.
Those said, I emphasise with unfamiliarity. Aussies laugh at me, but I've actually been learning Imperial measurements in my own time so I can chat with my American friends about weather, etc. If that sounds condescending I don't mean it to, it's genuinely hard. "26 degrees" means something to me, "79 degrees" is a step away from being useful. I also appreciate it's easy for me to say "move to Metric" given it's the system I use.
I'd argue though the potential benefits far outweigh the negatives though, as they did in Australia. Along with NZ, it's proof that the Metric system can be used in the unwieldy Anglosphere after all.
The Wikipedia article explains Australia's metrification (metrication?) process pretty well actually, including the myths that switching causes more road accidents, etc.