So, I co-wrote this book on virtual security and am a former VMware Cloud Solutions Architect. And I'll preface this advice by saying that, if you want to talk more in depth, feel free to ping me. First initial, last name at gmail will work. (The email I have attached to slashdot I glance at occasionally, but it gets almost purely spam and so I'd likely miss anything.)
From my perspective, the first question is which hypervisor to use:
- VMware is mature, you can get a free license for the base hypervisor (which is quite feature rich; this is no trial product) for up to 32GB per physical box, is widely used. If VMware remains as relevant in the future as it is now, it's actually a very solid skillset to have.
- If you have physical hosts over 32GB, VMware ceases to be free
- Some features require more advanced VMware stuff, including vCenter server, which isn't free - for example, VMware's live vm migration feature (vMotion)
- VMware is almost entirely closed on the internals; hypervisor is closed source (other than a not-useful-for-your-purposes "open source" bundle that contains their modified GPL code only); they have a bunch of APIs for internal functions (ie, tracking changed blocks on the virtual iscsi devices, for example), but those are generally restricted to partners; so if your students want to actually hack the virtualization layer, they can't. Then again, letting them do so wouldn't really be safe.
- On the other hand, VMware layers do have nice APIs that are reasonably accessible for doing non-internals stuff; things like powering VMs on and off, changing their allocated RAM and cpus, etc
- VMware has a nice set of tools, including CLI tools, which work well even with the free versions, that can allow you to move virtual machines in and out of specific hypervisors (not while the VMs are powered on), and into and out of VMware's desktop products (Workstation for Windows and Linux, Fusion for Mac). (google ovftool for the cross-platform CLI tool, for example; it can import/export to/from ESX, vCenter Server, Workstation, Fusion, and vCloud instances)
- VMware has a nice set of tools for snapshots and backups, even on the base hypervisor; for example, I have a personal ESX box at a provider and I use this tool to back up the VMs back and forth, which can be done from outside the OS without powering the VM down, and it's free.
- I found using some things I'd think of as mandatory for a lab environment (ie, thin provisioning) were just built-in on the VMware side and required a fair bit of extra work and added extra wrinkles
The virtual networking on VMware is dramatically more mature from my experience; my experience with Xen & KVM is now dated (it's been 2 years since I was in the thick of writing that book, which was the last time I was really in the thick of exploring the open-source hypervisor networking bits). I found that depending on the version of the hypervisor OS, which hypervisor, which kernel, which guest, etc, you could fall into all sorts of traps. I had some examples in the book where I showed, for example, generating and applying ebtables configurations to the host OS (the Xen Linux hypervisor OS) to block forged frames from coming across the bridge from one of the guest Linuxes, for example.
Compare that to the VMware side, you could in theory wire up everything to dumb hubs, even, and enforce network separation at the hypervisor layer with VLAN tags applied to the portgroups where you attach VMs. (Warning: not suggesting you blindly do that; but VLAN enforcement on the VMware side is fairly rigid if configured in a good way.)
My own book is a fun read for some of these concerns, although Haletky's book is probably the canonical work on the subject. (Although it is -slightly- dated from being a bit old, it is still a wealth of great information, and it was a huge help to me as a primer when I first joined VMware.)
Depending on how far you want to deep dive, my second choice might be Xen+Eucalyptus; if you could front-end your hypervisors with Eucalyptus and build an internal cloud, you'd also get your students one foot on the road to playing devops with AWS. (There are plenty of VMware clouds out there, but I don't know of any offhand that have the equivalent of the AWS micro tier, which would let students even occasionally deploy their boxes to the net.)
One final consideration is that VMware actually gracefully does nested virtualization; you can run ESX inside ESX, and you can run Xen inside of ESX, and they generally function well. The Xen FAQ implies Xen supports it, but I'm unsure if Xen can nest VMware or KVM for variety; I can say from experience that I had VMware, with Xen inside it, with a guest OS inside that, all on my laptop just fine.
Good luck! This is super-fun. I will say: don't overlook the value of the actual virtualization layer experience! It is currently far harder to find solid virtualization & cloud engineers than it is to find a Linux admin. The rise of virtual appliances and infrastructure as an extension of code makes me feel like the devops & virtualization skillsets will remain in strong demand, and operating systems may be simply seen as containers for applications.