The IMR is never a good statistic to use because in almost every other country in the world, a baby has to meet certain conditions to be declared official "alive", and as such, those conditions have to be met to the baby to end up dying.
The US considers any baby that came out of the womb alive, and so any baby that dies adds to the IMR.
Lastly, we have a lot of minority groups that tend to be poor and have a very high rate of premature births. This is a double-edged sword in which they can't afford insurance and they're more likely to have a problem, whereas in Europe and most other countries with health care, societies are still very homogeneous and there isn't a high rate of prematurity. Sweden has 1/2 the premature birthrate of the US.