Experiences with Replacing Desktops w/ VMs? 442
A user asks: "After years of dealing with broken machines, HAL incompatibility, and other Windows frustrations, I'd like to investigate moving to an entirely VM-based solution. Essentially, when an employee comes in in the morning, have them log-in, and automatically download their VM from the server. This gives the benefits of network computing, in that they can sit anywhere, if their machine breaks, we can instantly replace it, etc, and the hope is that the VM will run at near-native speeds. We have gigabit to all of the desktops, so I'm not too worried about network bandwidth, if we keep the images small. Has anyone ever tried this on a large scale? How did it work out for you? What complications did you run of that I probably haven't thought of?"
Re:Um, wouldn't a ... (Score:5, Informative)
Look at LTSP.ORG (Score:5, Informative)
I work like that now, but 2000 miles away (Score:3, Informative)
This works great, with one major caveat. If the network starts stuttering, performance of remote desktop and citrix both suffer badly. Otherwise, the benefits are great: much reduced amount of sensitive data on laptop, access to a higher performance office machine, less app latency when talking to 'local' databases 2000 miles away.
The way we do it... (Score:4, Informative)
My experience... (Score:3, Informative)
However, for the context that you are speaking about, I would take the advice of individuals below and look at Citrix or roaming profiles.
Re:The way we do it... (Score:3, Informative)
The desktop is still a problem though.
Re:Please, god, no. (Score:4, Informative)
Re:No 3D (Score:1, Informative)
Exactly. A good test to see how the speed on how the proposed system would perform is using terminal services (obviously on a remote computer). Even with gigabit ethernet the speed is going to drag on many applications. IMHO, if the author is running a Windows network, he or she should just use remote desktop and then standardize each computer with a default installation. Personal data will be saved on the server while all the heavy lifting software will be done locally (with the option of remotely executing high speed aps on other powerful remote machines). To speed up the updating of computers, if they are all identical he can simply use ghost software to copy hard drives. If a computer has a software problem, swap the hard drive with a previously ghosted copy (of the standard installation) and then ghost copy to that hard drive later--or ghost by network. I've done this and it works. It also makes it really easy to train junior sysadmins. Just make sure you don't allow users to store anything other than temporary files on the local systems HDs.
Re:Why not just use sunrays? (Score:2, Informative)
And no, I don't not work for/am not in any way affiliated with Sun Microsystems - I just really like their product.
Sampizcat
Three different takes on this (Score:5, Informative)
Here's what you should be thinking about:
- Get some kind of desktop management suite like Altiris. You can push software deployments easily, and it's very easy to lock machines down to the point where users can't fsck them up. I've consulted for companies that do this with hundreds of desktops and it's a very robust, reliable system.
- Go with a thin client setup like Citrix or Terminal Server. Users run nothing on their local hardware. Instead, everything runs on the big server. Downsides are similar to VM's (thin clients are notorious for very lightweight support for anything but the most basic sound and graphics) but you are at least spared the massive network thrashing of hundreds of users logging on and pulling down VM images at 8AM every morning.
- If it's users messing up machines that you're worried about, you might want to consider a solution by Clearcube. They take away everything except the keyboard, mouse, and monitor. The guts of the PC reside in a server rack in what is essentially a PC on a blade. The blades are load balanced and redundant, so swapping them out is a breeze. And users *can't* load software on them because there's no USB ports, no floppy drive...nothing! Unless you allow them to download it from the Internet, *nothing* is going to get on those machines if you don't want it to.
VM's make sense for server consolidation. I don't think they've yet gotten to the point where desktops run on them as a form of protection or reliability. There's too many other solutions that work better and have fewer downsides. The problem here isn't Windows per se, it's the fact that your workstations aren't locked down properly to prevent your users from doing stupid stuff in the first place. Fix that and suddenly you'll find a Windows workstation environment isn't the hassle it once was.
Re:Why not just use sunrays? (Score:5, Informative)
Come on, you're going to have to give some additional information than that. We use Sun Rays quite a bit in our classrooms and labs and if you have the bandwidth and a good server on the other end, you're in the money. Sessions can be keyed to an access card and will follow you around the campus. If a Sunray breaks down, just swap in a new one and the session continues exactly as you left off. Pull your card, come back in a week, and pick up exactly where you left off. Everything resides on the server. No maintenance required at all on the client side.
What version of the Sun Ray server software were you using that made it so "sucky"? From my experience, they worked great for us. The only downside we had is that streaming video over Citrix to the Sun Rays didn't work so hot. However, streaming video natively from the Sun Ray server to the thin clients worked fine so the problem there was probably with Citrix Metaframe.
Sun has also recently upgraded the Sun Ray thin clients so they have gigabit ethernet, plus they now hsve a more complete end-to-end solution that will allow you to run Windows apps on your Sun Ray (in addition to all the Solaris/Unix apps) thanks to their Tarantella purchase. You'll still need some Terminal Server licenses, but you'll save on the Citrix.
You could try calling the local Sun reps and see if they'll give you a demo. They did that for us - drove 6 hours to our workplace and set up a server and clients to demonstrate it for us.
Re:Citrix (Score:5, Informative)
Citrix (or another similar product) is exactly what he should be looking into. Downloading entire disk images over a network is just a pain in the ass everytime someone boots. However Citrix isn't the solution to all things, yet it beats VMs for most practical applications.
See, that's the big negative point in the entire setup. The bootup time is a pain in the neck, but people can live with that easily. They'll fetch their cups of coffee, have the morning conversation with coworkers and will return about 10 minutes after their machines have booted up. The real issue is the server getting hammered every morning, slowing these boottimes as more machines get added to the network.
I can hear it now: set up a second server, set up a third... etc etc. Yes, set up a bunch of servers that do nothing all day but hand out images, and don't forget about the backup servers (you don't want one of those servers to crash in the morning taking out the entire accounting department). I'm seeing an entire rack of machines at this point doing nothing but handing out images, wired up to really expensive network gear, doing nothing really useful. Don't get me wrong in this last statement, the usefulness of this construction is that you can easily exchange pc's and images not having to worry about hardware, software installed on each users pc, etc. But there's a lot of more cost-effective ways to achieving something that works similar.
Take that budget for those image servers, and backup servers, VM-software licenses, and networkgear, and buy a single server and a good backup mechanism (or a backup server in failover). Spend some time on setting up profiles and think about what software is present on all machines. Take an image of every machine you install differently, and copy that to the server. Buy software like Citrix (or anything else resembling it) to have special applications available at one server (think backups here), and you have a pretty decent solution that doesn't hammer your network/servers every morning and gives you a headache by 10am because some people aren't getting their images.
I've seen the concept of VM images on a server, and I've seen people get bitten by it because they didn't forsee the amount of storage and network traffic involved. Most of these people didn't have a need for such an elaborate solution. Hell, I've seen half a serverfarm run vmware because "it was a good way to virtualize systems, and make things easily interchangable" while those people would've been much more satisfied with a "simpler" failover solution (note those quotes, denoting that failover also requires thought, but usualy ends up being a cheaper solution hardware wise).
On top of it all, using VMs for desktop operating systems uses up a lot of resources. You're running an operating system, that runs software that runs another operating system. Some would say that it's hardly noticeable, but why waste the resources? You'll make todays hardware run like last years, which for most applications is not an issue, but most likely you're going to run last years hardware like hardware from two years ago because you'd have to invest in new desktops for the entire company otherwise.
Let's talk mobility for a moment. Imagine your salesman with his laptop and flashy UMTS (or whatever standard they've cooked up) connection on the road. He's going to want to be able to check his mail on the road, so he'll have to get an image over a connection that can hardly manage streaming video... Nope, you're going to give him his operating system, install his software and pray to god he doesn't send too many large documents over that very expensive UMTS connection. That sort of starts breaking the principle of having images f
Re:No 3D (Score:5, Informative)
Amazing (Score:1, Informative)
Yet, you throw out basically worthless info. I am sure that they will be trialing it. But if others have done it, and offer useful info, they can also check out paths to take (or avoid).
To E1ven: Please try it out on a couple of different set-ups and let us know. It would be useful to see how it works with Xen (combined with qemu for the windows stuff).
PXE Boot (Score:5, Informative)
You can then just capture or encapsulate the computing session to an image file. It's not a full virtualized environment, as you still get the benefits of the cpu horsepower at the workstation, but if corruption occurs ou just roll back the session file. I think.
This is how Windows hibernation functions in a nutshell, just dumps RAM to a file I think. I haven't tried this in practice, but it should work.
Re:Amazing my left foot (Score:1, Informative)
In contrast, it is when a company locks down everything and is afraid to move forward with new ideas that dies (or nearly dies). For example when they start saying that the company should not change things, then they are in a death spiral.
Re:Um, wouldn't a ... (Score:1, Informative)
Are you using cheap (or onboard) NICs, cheap switches and long wires? Do the math:
100Mbps link over long wires + cheap NICs (like Realtek) + cheap switches (like Planet) = 10Mbps link
100Mbps link over short wires + 3com NICs + 3com switches = 100Mbps link
Re:And this would be an improvement how?... (Score:2, Informative)
We have a Linux server that runs Samba for roaming profiles to the current Windows laptops and this works OK as it does mean if a laptop dies the user has all their configuration stored on the server but unless the replacement machine is configured exactly like their old one (and the users do have various needs for software beyond just the basics so they often do differ) the roaming profile doesn't exactly work and there is a bit of fiddling.
With the VM setup the users are able to use their image on any machine (shortly even on the Macs) and it is theirs regardless so the roaming profile works well too. This also means that Windows only uses up a small part of their workstation so we can gang the quads together into a cluster and do some serious work. The best part is that each night we do an rsync of the home directories (to another server and external drives to be stored in a firesafe) which also contains their VM and so if they screw up their Windows system we can just copy back the one from the day before and all is well. Far better than Windows Restore which isn't entirely able to put a machine back into a previous state.
Finally, the price of all this destroys any other solution I can think of for running Windows apps in a largely Linux environment. The player is free, the Windows and Office licences we had already bought, Linux is free and we have got a 40 processor Opteron cluster available that effectively cost us nothing too because we needed to put desktops in to replace the laptops that some idiot thought would be a good idea when the company first got started. Every user has a local vmplayer on their Linux machine. They are getting dual 20" monitors which is better than the 15" laptop with a 17" monitor attached as the laptops can't drive anything better so they can run Windows on one monitor and have Linux on the other.
With the current situation they were all running lots of Linux apps using VNC to our few available Linux machines and lots of terminals (cygwin or putty) but had Windows because there is still a perception that we need Office, mostly PowerPoint, although I have made sure they all have OpenOffice. They were crying out for more compute power as the company grew so for not much money I was able to buy 10 Quad Opteron workstations to give them the power, dual monitors don't cost much now either and vmplayer gives them Windows on one screen and Linux on the other, or Linux on both if they don't need to be running Windows apps. They still have the laptops for presentations and mobile use but they don't need to use them every day which should prolong their lives so there is a saving there too. What's not to like?
VMware ACE (Score:4, Informative)
"With VMware ACE, security administrators package an IT-managed PC within a secured virtual machine and deploy it to an unmanaged physical PC. Once installed, VMware ACE offers complete control of the hardware configuration and networking capabilities of an unmanaged PC, transforming it into an IT-compliant PC endpoint."
Re:Citrix (Score:5, Informative)
Granted, for large network this solution is probably too expensive (we are
Re:Um, wouldn't a ... (Score:1, Informative)
funny but my p3 NFS server with a realtek nic can transfer files to my desktop over cheap ethernet cables, through a cheap 100Mbps switch, into a cheap realtek NIC at around 10-11 megaBYTES per second. That's pretty close to the maximum speed you'll see over any 100Mbps link. Long wires might make a difference, but cheap nics and cheap switches seem to work fine for me. Maybe the switch will be a lot more a limiting factor if more than one simultaneous transfer is going on (I use cheap switches for my LAN where normally only 2 machines are communicating at a time). I actually changed my nfs server to a realtek NIC because the 3COM 10/100 card refused to work in 10/100 mode, and always went to 10. The cheap realtek cards are wonderful, as they're dirt cheap, and work great in linux.
Re:Still Windows (Score:5, Informative)
You don't need to "carefully" do anything. Folder Redirection in Windows was created just for the task. It's a feature that was introduced with Windows 2000. Beyond that you can use SMS and custom office installs to have everything configured properly everytime someone logs in. Mandatory profiles ensure that everything stays clean and spyware free. Which weaknesses are you referring to?
Beyond that I'll go and say that this approach is bomb proof and by redirecting files on to the servers which requires surprisingly little overhead you ensure that when users float from machine to machine they have all their application preferences and data. Settings can very from machine to machine with different version of software and whatnot but again, SMS will fix that.
I think we can all agree this is not a good use of virtualization. It would be very resource intensive and a simpler PXE solution already exists. With PXE you don't even have to have all the same hardware, just the proper drivers. SMS will take it from there installing the rest of the third party apps whatever they may be. Can be done from start to finish in under 30 minutes which is about how long it takes to fully restore an image. Of course over a gigabit link the time might be reduced but Windows will take a good 10 - 15 minutes to install over the network so it wouldn't be unreasonable for everything else to take another 15 minutes depending on how much there is. I know in my basic setup with Windows and Office its about 20 minutes give or take depending on processing speed and quality of hardware.
Re:Inevitably (Score:3, Informative)
I'm amazed nobody has brought this up. Someone said you'd need twice the XP licenses. That isn't true if the host OS on each PC is Linux. VMware DOES have a client for Linux. I imagine it would be a lot more secure if Linux was the host, and you can customize all the startup scripts so you wouldn't have any unncessary overhead.
I'm not sure on the loading time differences between XP host and Linux host, but last time I ran VMware on my XP laptop, it was slower than shit. 1gb of ram, Turion 1.8ghz, it ran slow...... SLOW
Re:Look at LTSP.ORG (Score:1, Informative)
Windows licensing terms prevent this (Score:4, Informative)
Re:Citrix (Score:3, Informative)
Re:Inevitably (Score:5, Informative)
My friend had setup Windows for his girlfriend as guest OS under Linux host. He was using VMWare. His girlfriend was forced to use IE to access her University Intranet. Also she needed M$Office for documents from profs. The notebook was constantly plagued by malware/spyware/etc making it barely usable.
My friend installed Linux (Gentoo one) and VMWare Workstation. Inside the VMWare he installed the OEM Windows off the notebook. State of Windows - fully updated and with M$Office installed - was saved on backup image. In guest Windows, all work was done on SMB/CIFS drive of host Linux.
The only problem was video performance - e.g. macromedia flash animations at times were making the notebook to melt. Also there were some sporadic network problems - mostly attributed to poor Windows network stack implementation. (IOW, the network problems occurring with normal Windows installation under VMWare were occurring more often. E.g. Windows DHCP client was at times failing to get address from host Linux. That IE thing was at times failing to load pages properly or simply hanging. The usual WinXP/IE problems.)
Advantages were clear. Spyware/malware got to notebook? - recover from backup image. Something crashed? - data are most likely are Okay on host Linux hard drive. Also, gradually girlfriend ha been taught how to use Linux and how to get around the University Intranet with Linux and Firefox/Konqueror. Though most documents she used still required M$Office under Windows.
Parallels - Mac - 3D hwAccel. (Score:2, Informative)
Re:I respectfully disagree (Score:1, Informative)
I guess Linus, etc., would say it is to prevent hardware manufacturers from making closed source drivers, but that doesn't work anyway because either they just don't bother with a linux driver or they make a thin GPL shim which presents a consistent ABI to their closed driver like nVidia. In the end it just makes Linus more of a pain to use.
I respectfully disagree with you though, at my GF's request we have been moving her away from Windows 2000 and onto Ubuntu. Running (free!) VMWare Server on Ubuntu works great (apart from the occasional module recompile issue) and now her old Windows installation which we're still running in parallel during the switch is where it belongs - in a box. Windows has no business running on the metal
Re:No 3D (Score:5, Informative)
Java and
VMware has actually had 3D support for a while, but it's been painfully slow. The latest versions do make some attempt at using hardware 3D acceleration through the host operating system. I'm not sure how well at works, though.
Re:Um, wouldn't a ... (Score:4, Informative)
They even do audio decently, which is pretty nice. (and support USB peripherals beyond the kbd/mouse, etc.) Their only weak spot is video (and flash on web pages). Video could be fast if it used the right APIs, but the *only* program I've seen that does this is Sun's now defunct "ShowMe TV" app.
Re:No 3D (Score:5, Informative)
Re:I work like that now, but 2000 miles away (Score:2, Informative)
In fact, you can tunnel most network protocols over VPN....
Re:Um, wouldn't a ... (Score:2, Informative)
Xen/VMWare (Score:2, Informative)
On the other hand, VMWare is a bit sluggish. It's easier to set up virtual machines for but I've never used it where I didn't feel like it was sucking up system resources and slowing down both host and virtual machine. I also found that it seems to screw with the virtual machines clock a lot
ARDENCE is what you want!! (Score:1, Informative)
Re:Um, wouldn't a ... (Score:4, Informative)
I was introduced to the sunray in college. When better then 100 workstations are needed the cost and administration savings become very apparent. 100 * 40 watts (sunray and LCD) == 4000 watts..... 100 * 200 watts ( PC and LCD ) == 20000 watts. The new devices draw 10 watts of power. ( but cost a lot more )
Also, they have very good power managment and go to sleep quickly, saving more energy. They are also fanless, giving a much more pleasent work (or in my case bedroom) environment.
Re:Another Possible Solution (Score:2, Informative)
Synchronization (Score:3, Informative)
Try Unison [upenn.edu]. It caches the state of the last sync, so it's dramatically faster at startup. Under the hood, it uses the rsync protocol when it does need to transmit changes. Additionally, it's much more configurable than rsync.
I use Unison to sync/backup my home and work computers, including my music and photo collection as well as ~/bin, ~/perl, ~/.cshrc and ~/.emacs.
Our experiences (Score:3, Informative)
For remote users we use Cisco VPN to the remote desktop.
Citrix licensing is expensive but you should first rough out some numbers as to how much it costs to support the desktops individually versus the same tasks by one or 2 techs on one or two servers plus Citrix costs.
We are using VMs in our development and test environments on an older AMD 64 bit machine. It still bogs down after 3-4 Vm's are running so my advice is to by the biggest, fastest and most reliable box you can. Lots of memory, fast disks and memory and CPUs. Newer 64 bit hardaware would be sweet as you should be able to set up 32 bit OS's on it and support older apps without having to upgrade everything to 64 bits all at once. Make sure it is not 'cutting edge', rather if it is for critical apps make sure it is stable on the hardware side, even if you sacrifice a little speed. Think in terms of how mainframes do things.
HTH
Re:Citrix (Score:3, Informative)
Some old fashioned roaming profiles and ghost (or some other imaging solution) action would seem to be the perfect compromize. Local CPU gets utilized. Network traffic is minimal. Users get good performance. Users can move to any machine with the proper appliations installed. A machien can be rebuilt in under 10 minutes. Instead of maintaining an image for EVERY SINGLE USER, you only maintain an image for every *type* of workstation in your company. Honestly, I can't figure out how this solution was overlooked.
-matthew
Re:No 3D (Score:4, Informative)
Servers are much more likely to be network I/O bound, physical I/O bound, or CPU bound. VMware has done a lot of work to create accelerated network and disk drivers that communicate directly with the host VM software, avoiding the overhead of emulating hardware. Additionally, most software that runs on a server spends a lot of time processing things in user mode with occasional system calls to transfer chunks of data, so the VM overhead is very low for those applications.
Desktops are much more demanding on I/O. Applications with a GUI are constantly context switching between user mode and the kernel in order to update the screen, display pretty animations, get keyboard / mouse input, etc. Just drawing the graphics for a 1280x1024x24bpp display is an immense data transfer burden that has to be routed through the VM, often a few dozen pixels at a time rather than a bulk transfer. Even with accelerated drivers, the virtual framebuffer still has to be copied / multiplexed into the real one. Even small increases in latency are very user-perceptible, so a VM will seem more "sluggish" than a physical machine.
That's just for business applications. Once a user starts playing audio (it'll be uncompressed 44/48khz PCM data by the time it gets to the VM!), streaming video (no access to the hardware YUV conversion or scaling!), or trying to do anything that needs 3D acceleration, the full impact of the virtualization will be felt.
-- Ironically, the spell checker in Konqueror wanted to change "VMware" to "Vampire"
Re:Windows roaming profiles (Score:2, Informative)
Use a logon script to change their temporary internet files to a reasonable size. For some reason the temp internet files are in the roaming profile. By default, they're a percentage of the drive (usually gigs).
This one sets temp internet files to 128mb. It works for me but YMMV
Slashcode seems to have added some spaces, but you get the picture
Re:Um, wouldn't a ... (Score:1, Informative)
12 thin clients, works great until 4 people open Open office at the same time. I only have 4 gig of ram on the server so it all crawls to a snail as open office consumes all ram. (Please! Office suites need to be smaller than they are!)
Having seperate App servers significantly increase speed and maintainability. but this guy is talking windows, and doing thin clients under windows simply sucks. (Yes Citrix sucks compared to LTSP.. I maintain both and am citrix certified, citrix sucks!)
The cool part is that P-1 233 clients are as fast as overpowered dell junk the corperations are using today.
Small company, I Increased reliability and useability and cut IT costs by 60% by moving to LTSP. everythign just works and everyone has nice 19" lcd's on their desk, dead PC = 3 minutes of work swapping out the old pc with a new one by one of our low wage college students. Only idiots still use Microsoft in their company... well maybe not idiots, only IT incapable of learning new things...
Re:PXE Boot (Score:1, Informative)
Get a few people using a machine, and the local drive fills pretty quick (new outlook cache for every profile, sometimes multiple for a single user if windows does that thing where is gets confused and keeps creating a new local user profile periodically on login for the same friggin user. ie username, userna~1, userna~2,
It's also a pain if the outlook cache gets corrupted (frustrated user, "Why am I getting months old messages marked as new in my inbox?") - in which case you'll have to track down the cache file and delete it.
Re:Windows roaming profiles (Score:3, Informative)
(MOD UP) (Score:3, Informative)
Also check this registry setting:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList
Consider changing the ProfilesDirectory to a mapped drive network share. If your network is fast enough this has the added benefit of having no profiles stored locally at all.
Downside is you'll have to pre-emptively create all the folders on that drive because LOCAL SYSTEM won't have permission to create the folder at first logon to a machine. But it'll detect if it's there and think you've logged on before.
Make sure you use Samba as the backing for a system like that though because you will need to play with oplocks settings so Windows doesn't bitch about loading the NTUSER.DAT hive from a network share.