Second is the pretty-good reason: compatibility and correctness. You can definitely have multiple major versions (e.g. the runtime associated with VS2008 and 2010) installed simultaneously, and I think you might be able to have multiple patch versions of the same library installed simultaneously. I think the former is true in Linux too (libfoo.so.1.0.0 vs libfoo.so.2.0.0,
Well, you're not likely to have multiple versions of the C runtime installed, because, in most if not all UN*Xes, the C runtime is part of the equivalent of kernel32.dll (libc.so, libSystem.dylib, or whatever it's called).
But, yes, you can have multiple "major" versions of libraries present. The SVR4 shared library mechanism, upon which the Linux and *BSD shared library mechanism are based, and the SunOS 4.x shared library mechanism upon which the SVR4 mechanism is based, gives libraries "major" version numbers, which change when the library ABI changes in a binary-incompatible fashion, and "minor" version numbers, which change when the library ABI changes in a way that preserves binary compatibility with older library versions but might add features (routines, flags to routines, etc.) that, if used, won't work with those older versions.
However, if your application uses libfoo version 2, but it's linked with a library that uses libfoo version 1, that's a problem. (Replace "a library" with "libpcap", and replace "libfoo" with "libnl", and you have one of the problems that makes me want to have libpcap on Linux talk directly to netlink sockets without the "help" of libnl, but I digress....)
but the latter isn't so much. It may well be that Program A installs version 1.0.0 and Program B installs version 1.0.1239, where on Linux the latter would likely be packaged to upgrade the former.
If libfoo is done correctly, any program linked with version 1.0.0 should Just Work with version 1.0.1239, and Program B should only upgrade to 1.0.1239 if there's a bug in 1.0.0 through 1.0.1238 that breaks Program B so it requires 1.0.1239 or later, and Program A should just require 1.x.x and not install 1.0.0 if 1.0.1239 is installed.
If you take the Linux approach, then programs which rely on the old behavior of the buggy code will break. This is sometimes good (e.g. bad security-related fixes), but is often not. Or it doesn't have to be a bug fix, it could just be some behavior change within the specification. By keeping multiple versions around, the Windows approach keeps the individual programs happier.
How you weight these various advantages and disadvantages is up to you. I'm not really trying to argue that the Windows approach is better, just explain why it is as it is and give a fair description of what goes on.
Yes, that's the question of the extent to which the real "specification" upon which clients depend on is the official specification or the full behavior of the implementation, and the extent to which you're willing to tell developers of code that doesn't fit the former specification but fits the latter specification to go pound sand if you "break" their code. Sometimes you end up not telling them to go pound sand, e.g. the "7090 compatibility mode" in the IBM 7094 (in which mode the index number field in instructions is interpreted not as an index register number but as a bitmask with bits corresponding to 3 of the index registers, with all the index registers specified by the bitmask being ORed together to generate the index) or the hacks in various OS X libraries in which the library detects that program XXX is using the library and falls back on the old buggy behavior (I think Raymond Chen's "The Old New Thing" has examples of similar hacks on Windows).