You realize what you are saying is equivalent to saying a new windows app needs to be test on every version of windows, and on every machine combination, right?
As the main build guy at a small software company, I can attest that this is necessary.
We run automated testing on XP, Vista, Windows 7, and Windows 8. As far as "every machine combination," yes and no. If your software is built for 32 and 64 bit systems, you need to test XP32, XP64, Vista32, Vista64, Win7-32, Win7-64, etc.
We also run on Linux and test on every distribution we support. Here, it's mainly glibc differences that cause the breaks. We build to the lowest common denominator, and you'd be surprised how often something works fine on RHEL4 but is broken on RHEL6 due to library differences.
We have seen lots of issues from the same code that are only reproducible on one version of Windows or one version of Linux.
If you're doing it seriously, test seriously or your product will suffer.