It's not the ambient temperature of air that's key here, it's the ambient temperature of space, which is about 2,7K.
All objects are constantly radiating energy and receiving energy back from other things that are radiating. When two objects in radiative exchange are roughly the same temperature, this balances out. But when one is hotter than the other, the hotter one loses more energy than it takes in, and vice versa. And it's not just a little difference - radiative heat loss is proportional to the absolute temperature to the fourth power, that's a pretty big exponent. So when you're exchanging energy with space, which is so cold that it takes very sensitive instruments to be able to measure *anything*, well, that heat is simply lost.
You can see this effect for yourself by noting how cloudy nights are usually warmer than clear nights. Clouds are cold, but they're not as cold as space!
The effect of the combination of radiation, absorption, and reflection, with different band peaks for each phenomenon, manifests itself in atmospheres as a greenhouse effect (positive or negative) versus the radiative equilibrium temperature.