Besides those numbers are for both winter and summer olympics combined, let's look into more details.
Sweden: 191 gold, 10 mln inhabitants
Hungary: 167 gold, 10 mln inhabitants
Switserland: 91 gold, 8 mln inhabitants
US: 1063 gold, +300 mln inhabitants
It is rude and unnecessary to call Americans fat and poor at sports, but in order to compare it with other nations one should not take grand totals.
(And yes, I hand-picked those examples but on the other hand I left out Norway with 5 mln inhabitants and 107 gold medals at the winter olympics vs US 87)