I agree with everything you said, I use VMs too. There's another benefit of using VMs: you have a frozen environment per version. I work on software that releases a couple of versions a year and every 2 years or so a big version jump. With these big version jumps we e.g. upgrade UI controls and introduce necessary breaking changes. When I start on these major version jump versions I move to another VM, cloned from the original. This leaves the previous version's VM in tact so I can fix bugs in that version's VM and alter whatever I need to (controls, libraries, IDE instances even) in the new VM for the new version. I don't have to worry I change something for the new version that breaks the older ones (e.g. uninstalling libraries no longer needed for the new one).
This alone is the key benefit for me for using VMs: no worries changes made to newer versions affect older versions. I can go back to an older version's VM and use the tools used for that, build using the build tools for that version and *know* everything needed for that version to build is there.