Slashdot Log In
Virtualizing Cuts Web App Performance 43%
Posted by
kdawson
on Thu Mar 29, 2007 07:29 AM
from the price-to-pay dept.
from the price-to-pay dept.
czei writes "This just-released research report, Load Testing a Virtual Web Application, looks at the effects of virtualization on a typical ASP Web application, using VMWare on Linux to host a Windows OS and IIS web server. While virtualizing the server made it easier to manage, the number of users the virtualized Web app could handle dropped by 43%. The article also shows interesting graphs of how hyper-threading affected the performance of IIS." The report urges readers to take this research as a data point. No optimization was done on host or guest OS parameters.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Virtualize this (Score:3, Insightful)
Re:Virtualize this (Score:5, Interesting)
You still take performance hits, but if you can scale your system by just adding cheap commodity systems, that works. Plug it in, boot it off a CD, and let the Cluster take control.
Parent
holy cow am I a nerd (Score:5, Funny)
I started getting aroused as I read your post. This is highly disturbing.
Parent
Re: (Score:2)
Re: (Score:3, Insightful)
No.
It's no different from focusing on any other major corporate product. ASP.NET isn't going away anytime soon and it's not going to fall over anytime soon.
It's run by Microsoft. Despite what the ranting morons around here think, Microsoft is not going away, IIS is not an insecure mess and it's actually a really good platform for web applications.
Now, that being said, use the best tool for the job, otherwis
Well, (Score:5, Insightful)
Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.
I especially love the idea that running different types of server virtualized on the same machine is a good idea; the idea of virtualization of multiple servers is to distribute the load. If you have, say, ftpd, httpd and mysqld running as their own virtualized systems, they will all get hit *simultaneously*.
Again. Duh.
Re:Well, (Score:5, Insightful)
Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.
Well, I think the point was that he attached an actual number to the amount of the performance hit, which is relevant. That's called research; quantifying and proving that which seems 'obvious'.
Parent
Re: (Score:3, Interesting)
But really. If you've got the money for the extra hardware to maintain performance, I say go for the virtualization, if only to make yout IT guys' lives easier (happy IT is useful IT).
But the conclusions don't match the research (Score:3, Interesting)
> These results indicate that a virtualized server running a typical web application may experience
> a 43% loss of total capacity when compared to a native server running on equivalent hardware.
This may lead to people believing that virtualiz]ation just isn't worth the advantages. The key problem is that there are several virtualization schemes. Off the top of my head, I can list:
* Xen
Re:Well, (Score:5, Insightful)
Here's a question: what is more available: hardware or skilled system administrators? Obviously hardware.
Here's a common scenario: you've set up a system to provide some useful package of services. How do you let other people duplicate your success? (1) tell them what hardware they need and (2) have them install and configurethe software on their hardware. Guess which item involves the most cost in the long run?
The hardware is easy; the greatest barrier and cost is the process of installing and configuring the software. That's one place a virtual machine is worth considering in producation systems. You aren't going to use something like VMWare in one-of-a-kind production systems. You're going to use it when you need to clone the same set up over and over again. This is very attractive for application vendors, who spend huge amounts of support on installation and tracking down compatibilty conflicts.
Another application would be an IT department that has to support dozens of more or less identical servers, especially if they are frequently called upon to set up new servers. If I had a choice, I'd use Linux virtualization on a midrange or mainframe, but if those servers must be Windows servers, then I'd be looking at some kind of cluster with SAN. This is not really my area of expertise, but we're talking high end boxen for x86; if the typical server didn't need 100% of box, then I have three choices: waste server bandwidth (expensive), force groups to share servers (awkward and inflexible; what if I have to separate two groups joined at the hip?), and virtualization.
Naturally if you are virtualizing production servers, you need to scale your hardware recommendation up to account for VM overhead.
What would be very interesting is a study of the bottlenecks. If you are considering a system with certain characteristics (processor/processors, memory, storage/raid etc) and you have X dollars, where is that best spent?
Parent
Re:Well, (Score:5, Insightful)
Parent
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
In fairness, a 43% performance hit is a bit more than "a bit". It's cutting performance nearly in half.
I agree with your overall sentiment (a virtualized system is going to be slower than the same system running on real metal, by definition) but 43% is certainly a higher figure than I would have expected.
Bogus Test (Score:5, Informative)
Re: (Score:2)
Re:Bogus Test (Score:5, Insightful)
VI3 is actually a suite of products. At the heart is VMware ESX Server [vmware.com], which is actually an operating system in its own right: it runs "on the metal," without having Windows or Linux installed already on the system. It also has a service console operating system which looks suspiciously like a *NIX style operating system, so you can SSH directly to the system, cd into your
However, as a pretty damn safe rule of thumb, no system is going to run faster on equivalent hardware after being virtualized. In a prior job where I was often asked to provide development/test systems, I got phone calls from a lot of people who were bitten hard by the virtualization bug. Whenever someone brought up any issue having to do with infrastructure, no matter how odd or off the wall, they wanted to push virtualization as a solution. I had to explain to them that if your problem is that a web server is slow, the answer isn't to install VMWare server on it, set up two host operating systems, and say, "There! Now I have two web servers." You'd be surprised how pervasive that sort of thinking is, even among people who should patently know better.
Another useful guideline: various types of services are impacted differently by being virtualized. Generally, the best candidates for virtualization are ones that spend a lot of time idle. This is actually more common than you might think - people need a server set up for something, can't put it on a pre-existing system for security/compatibility reasons, so they go out and buy a new system which is ten times more powerful than they need. You can put a lot of these kinds of systems on a single, reasonably powerful ESX server. On the other hand, systems that heavily tax available hardware, especially I/O, are usually much harder to deal with.
Parent
Re:Bogus Test (Score:5, Informative)
In the end, the tweaked RHEL that you interact with (ssh, scp) is not the hypervisor, but a VM with special tools that can manipulate the hypervisor.
Parent
Re: (Score:3, Interesting)
Re:Bogus Test (Score:5, Informative)
One of the other things we prototyped and deployed was 'site services packages' - get GSX (now VMWare Server), stick it on a pair of 2U servers, and attach a storage array to both of them. Then create your 'template' fileserver, DHCP server, print server, proxy, that kind of thing and deploy them to this package. It worked very well indeed - you get a whole new order of magnitude on stability (although to be fair that's in part because we through away the crappy workstations that were doing the 'low intensity' stuff) and was extremely managable, and trivially replacable in the event of a hardware failure.
Performance? No, VMWare isn't that great on performance - whilst it's not bad, in an ideal situation, fundamentally what you are doing is introducing an overhead on your system. And probably contention too. But it's really good at efficient resource utilisation, easy manageability and maintainability.
As an experienced sysadmin, my reaction is screw performance. Let's start with reliable and scalable, and then performance just naturally follows, as does a really high grade service.
Proactive laziness is a fundamental of systems admin. Your job, is essentially to put yourself out of a job - or more specificially, free up your time to play with toys. The best way to do this is build something stable, well documented and easily maintainable. Then your day consists of interesting stuff, punctuated by the odd RTFM when something doesn't work quite right.
Parent
Re: (Score:2)
It's
Re: (Score:3, Insightful)
This is VMware Server and not ESX Server (Score:5, Informative)
Re: (Score:2, Informative)
Doesn't VMWare ESX run on some modified Red Hat version?
Also, we run ESX in our production environment, when we stress tested a web application running on IIS and with ASP/VB, the ESX machine couldn't give us more than 10 transactions per second (there was one single VM running on ESX). ESX was crawling.
T
Re: (Score:2, Informative)
According to Wikipedia [wikipedia.org], "VMware ESX Server uses a stripped-down proprietary kernel (derived from work done on Stanford University's SimOS [stanford.edu]) that replaces the Linux kernel after hardware initialization. The Service Console (also known as "COS" or as "vmnix") for ESX Server 2.x derives from a modified version of Red Hat Linux 7.2. (The Service Console for ESX Server 3.x derived from a modified version of Red Hat Enterprise Linux 3.) In general, thi
Re: (Score:3, Informative)
Re:This is VMware Server and not ESX Server (Score:4, Insightful)
Parent
This has been my experience too (Score:4, Interesting)
That said,
I use a windows vmware session under linux for those times I have no choice, and it works just fine network-wise as a workstation.
Re: (Score:2)
Re: (Score:3, Funny)
Oh yeah? Well my Johnson is longer than yours, and my son can beat up your son.
Sounds about right (Score:2, Informative)
That
single data point is correct (Score:3, Insightful)
While I can't seem to find all the information on the SC1420, it appears as though this product uses processors from the Prescott generation of Intel CPUs. Some chips from this group support "Vanderpool", Intel's hardware virtualization solution, but not all do. The presence or absence of this feature could greatly impact the performance penalty faced by operating a virtualized computing environment. Further, Intel's new Core2 based CPUs feature a hardware virtualization implementation which may have vastly different performance characteristics. AMD's K8 family supports hardware virtualization as well. I'm excited about their new line of CPUs based on the K10 (Barcelona) core, which feature "NestedPageTables," which are supposed to greatly reduce overhead by doing memory translations in hardware instead of in software by the hypervisor.
All I'm really trying to say is that this article really is only a single data point. I wouldn't let their results influence your overall view of virtualization in any way...
Re:single data point is correct (Score:5, Informative)
With paravirtualised devices, or devices that are virtualisation-aware, a VM can be within 10% of the performance of a real machine quite easily. Without I'm surprised they even got to 57% of native performance for web applications.
Parent
Pointless test? (Score:4, Insightful)
Re: (Score:2)
Re:Pointless test? (Score:4, Informative)
Parent
Re:Pointless test? (Score:4, Insightful)
You can also get better management control of resources, preventing one site from eating up all available resources on the box.
That's not to say there aren't a million good reasons to use virtual servers in apache, just to point out that virtualising web hosts is not, by definition, a daft idea.
Parent
Re:Pointless test? (Score:4, Informative)
Thus, AssignUserId should NOT be used. SuExec can be used, of course, but that has its own limitations.
Personally, I give users their own Apache processes on their own port (>1024) and use a reverse proxy. I make a living on it.
Parent
Re:Pointless test? (Score:4, Informative)
If you have a real need to run 100 separate Apache instances, then you'll want something much higher-level than VMWare. For us, that would be a FreeBSD jail, where each instance would get its own chrooted home directory and IP address. That way, you're not allocating resources to 100 little-used OS images; each shares from the same memory and hard drive pool. Jails are slightly limited in that I'd like a way to limit CPU and memory allocation, but in practical application this really works very well today.
Parent
Re: (Score:3, Informative)
Of those, only the last
Hidden advertisment (Score:2)
Bad data, bad setup (Score:5, Insightful)
1) As others have pointed out, they should be running on ESX to get best performance.
2) Physical machine was a dual-proc. How many processors did they assign to the VM?
3) Physical machine had 2GB memory. They assigned 2GB to the VM!! Vmware will take 256MB of this
for itself, so that 2GB visible to Windows will be being swapped.
4) How many disks did the physical machine have, and what was on them?
If e.g. the physical machine had two disks, the VM should have been given two disk files, with each file being placed on a different physical spindle.
You get the picture.
"Duh!" moment (Score:5, Insightful)
Web servers are mostly memory and CPU bound which would give one the impression that they would be great candidates for virtualization. However, VMWare Server is not the solution; network I/O is not good on Server. Typically your results would be maybe 75% of the actual physical speed on a "passthrough", less on a NAT. It depends a lot on how your network is set up, not to mention the abilities of the physical machine.
The best solution is Virtual Infrastructure (used to be ESX). That product tackles most of the failings of VMWare server and fixes them. The only exception is that I still wouldn't run anything I/O heavy on VI. SQL's a no-no. Also, if you're not getting the performance from a single web server that you expect, you can easily throw up more web servers. Now, obviously you might get into M$ licensing issues, but that's why you run your web services on Apache
Use Virtuozzo (Score:2)
http://openvz.org/ [openvz.org] - it does a much better job of virtualizing IMO. The only minus is that all VM's have to use the same kernel version.
Fast Virtualization: Xen, KVM, Virtuozzo, GSX, ESX (Score:3, Insightful)
These results are pretty much as expected (Score:3, Informative)
http://www.cl.cam.ac.uk/research/srg/netos/papers
Top of page 9 has a chart comparing native Linux, Xen, VMWare, and UML for different workloads. They show VMWare degrading performance by over 70% for SPECWEB 99.
Web applications are OS intensive; while VMWare is quite good at pure CPU-bound tasks, it has to perform a lot of emulation whenever you are running inside the OS. So it will stink at anything with lots of small IO, lots of metadata operations, or lots of process creation/switching. For example, VMWare shows a whopping 90% slowdown for OLTP database workloads, according to the Xen paper, and it really isn't surprising. The OS microbenchmarks in the above paper (page 10) show that VMWare has abysmal performance for things like fork(), exec(), mmap(), page faults, and context switches.
Basically, Xen doesn't have to emulate the OS, because they make modifications to the OS. VMWare does dynamic binary rewriting (think fancy emulation) to run an unmodified OS; they therefore pay through the nose in performance overhead for OS-intensive workloads.
This is comparing single to dual proc (Score:3, Insightful)
VMWare Server, the free edition only emulates a single processor environment for your virtualized host.
VMWare ESX or whatever they are calling the expensive thing today, has the ability to give your virtualized host multiple processors.
So it's not surprising that it could only handle half the load, it only had half the processors.
We don't do virtualization for heavy use environments. We do it because different business groups don't want to share servers... that is, they can't agree on maintenance windows, etc.
Re: (Score:2, Informative)
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:2)
It's a little early here and my vision was a bit blurry, it is 390 not 350!