Whether something feels hot or cool isn't a good indicator. That varies greatly depending on venting, and size of the housing. So your tiny little cell phone charger might *feel* just as warm as an old, *large* transformer, while consuming several times as much power.
Well duh, of course I'm not comparing an old-style wall wart in front of a fan to a modern wall wart in a closed box. Anyway, you would have to compare temperature difference (relative to environment) multiplied by surface area. Since most switched-mode wall warts are much smaller than iron-core transformers with the same power, the evidence that switched-mode wall warts waste very little electricity only becomes stronger: small area cool to the touch versus large hot area.