You're talking about thermal imaging. That's not how IR used for night-vision works. Your IR remote control doesn't shoot a jet of warmth at the TV. It's just a spectrum of light slightly outside of what we see.
Actual IR cameras work so well for finding people because of what's REFLECTING the IR light. Synthetic materials reflect differently to the sorts of things you find in the wild. Additionally IR is useful for marking friendlies in such a way that people without IR gear can't see.
Modern night-vision goggles use a combination of low-light sensitive cameras, IR imagery and image processing to enhance the image. I'm not actually sure if you'll find thermal imaging in use at all for combat situations.
Keeping that in mind, because the images are often of such low quality the concept of bending light around an object- even with large distortion errors - works very well. You're merely adapting the entire cloak to suit the surrounds, which is exactly what modern techniques for hiding from IR cams involves, except you disguise yourself with the surrounding foliage/debris/whatever. Same materials, same colour, same IR reflectivity. Fooling our eyes in daylight is going to be a little harder.