Forgot your password?
typodupeerror
Operating Systems Software

Virtualizing Cuts Web App Performance 43% 223

Posted by kdawson
from the price-to-pay dept.
czei writes "This just-released research report, Load Testing a Virtual Web Application, looks at the effects of virtualization on a typical ASP Web application, using VMWare on Linux to host a Windows OS and IIS web server. While virtualizing the server made it easier to manage, the number of users the virtualized Web app could handle dropped by 43%. The article also shows interesting graphs of how hyper-threading affected the performance of IIS." The report urges readers to take this research as a data point. No optimization was done on host or guest OS parameters.
This discussion has been archived. No new comments can be posted.

Virtualizing Cuts Web App Performance 43%

Comments Filter:
  • Virtualize this (Score:3, Insightful)

    by Anonymous Coward on Thursday March 29, 2007 @08:35AM (#18526789)
    That is all very well, but we all KNOW apps slow down when we run them in a VM. What difference does it make to the average n00b who wants to watch funny videos [digg.com] and check their email? Anyone using computers for serious numbercrunching obviously won't virtualize anyway. No big deal
  • Well, (Score:5, Insightful)

    by Fordiman (689627) <fordimanNO@SPAMgmail.com> on Thursday March 29, 2007 @08:37AM (#18526807) Homepage Journal
    Duh.

    Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.

    I especially love the idea that running different types of server virtualized on the same machine is a good idea; the idea of virtualization of multiple servers is to distribute the load. If you have, say, ftpd, httpd and mysqld running as their own virtualized systems, they will all get hit *simultaneously*.

    Again. Duh.
  • by Visaris (553352) on Thursday March 29, 2007 @08:48AM (#18526923) Journal
    Dell Poweredge SC1420 with dual Xeon 2.8GHz processors

    While I can't seem to find all the information on the SC1420, it appears as though this product uses processors from the Prescott generation of Intel CPUs. Some chips from this group support "Vanderpool", Intel's hardware virtualization solution, but not all do. The presence or absence of this feature could greatly impact the performance penalty faced by operating a virtualized computing environment. Further, Intel's new Core2 based CPUs feature a hardware virtualization implementation which may have vastly different performance characteristics. AMD's K8 family supports hardware virtualization as well. I'm excited about their new line of CPUs based on the K10 (Barcelona) core, which feature "NestedPageTables," which are supposed to greatly reduce overhead by doing memory translations in hardware instead of in software by the hypervisor.

    All I'm really trying to say is that this article really is only a single data point. I wouldn't let their results influence your overall view of virtualization in any way...
  • Pointless test? (Score:4, Insightful)

    by geoff lane (93738) on Thursday March 29, 2007 @08:52AM (#18526953)
    Come on! You run virtualised web servers because 99.9% of all web servers are idle at any given time. So you put 100 on a server. The customer doesn't see any worse performance with their 3 hits a week page and the ISP makes more money/server.

  • by duncanFrance (140184) on Thursday March 29, 2007 @08:56AM (#18526989)
    There's quite a lot wrong with their setup.

    1) As others have pointed out, they should be running on ESX to get best performance.
    2) Physical machine was a dual-proc. How many processors did they assign to the VM?
    3) Physical machine had 2GB memory. They assigned 2GB to the VM!! Vmware will take 256MB of this
    for itself, so that 2GB visible to Windows will be being swapped.
    4) How many disks did the physical machine have, and what was on them?
    If e.g. the physical machine had two disks, the VM should have been given two disk files, with each file being placed on a different physical spindle.

    You get the picture.
  • Re:Well, (Score:5, Insightful)

    by Mr. Underbridge (666784) on Thursday March 29, 2007 @09:17AM (#18527157)

    Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.

    Well, I think the point was that he attached an actual number to the amount of the performance hit, which is relevant. That's called research; quantifying and proving that which seems 'obvious'.

  • "Duh!" moment (Score:5, Insightful)

    by Thumper_SVX (239525) on Thursday March 29, 2007 @09:21AM (#18527215) Homepage
    I agree with many of the commentators here that this is pretty obvious. We use virtualization a lot, but also realize its limitations. For example, we don't run SQL or anything heavily transaction or I/O bound. CPU utilization is usually not a problem; virtual machines perform as well as their physical counterparts in most instances unless you have a lot of CPU intensive virtual machines running.

    Web servers are mostly memory and CPU bound which would give one the impression that they would be great candidates for virtualization. However, VMWare Server is not the solution; network I/O is not good on Server. Typically your results would be maybe 75% of the actual physical speed on a "passthrough", less on a NAT. It depends a lot on how your network is set up, not to mention the abilities of the physical machine.

    The best solution is Virtual Infrastructure (used to be ESX). That product tackles most of the failings of VMWare server and fixes them. The only exception is that I still wouldn't run anything I/O heavy on VI. SQL's a no-no. Also, if you're not getting the performance from a single web server that you expect, you can easily throw up more web servers. Now, obviously you might get into M$ licensing issues, but that's why you run your web services on Apache :D
  • Re:Well, (Score:5, Insightful)

    by hey! (33014) on Thursday March 29, 2007 @09:32AM (#18527311) Homepage Journal
    Well, it's not a surprise, but it's probably worth quantifying.

    Here's a question: what is more available: hardware or skilled system administrators? Obviously hardware.

    Here's a common scenario: you've set up a system to provide some useful package of services. How do you let other people duplicate your success? (1) tell them what hardware they need and (2) have them install and configurethe software on their hardware. Guess which item involves the most cost in the long run?

    The hardware is easy; the greatest barrier and cost is the process of installing and configuring the software. That's one place a virtual machine is worth considering in producation systems. You aren't going to use something like VMWare in one-of-a-kind production systems. You're going to use it when you need to clone the same set up over and over again. This is very attractive for application vendors, who spend huge amounts of support on installation and tracking down compatibilty conflicts.

    Another application would be an IT department that has to support dozens of more or less identical servers, especially if they are frequently called upon to set up new servers. If I had a choice, I'd use Linux virtualization on a midrange or mainframe, but if those servers must be Windows servers, then I'd be looking at some kind of cluster with SAN. This is not really my area of expertise, but we're talking high end boxen for x86; if the typical server didn't need 100% of box, then I have three choices: waste server bandwidth (expensive), force groups to share servers (awkward and inflexible; what if I have to separate two groups joined at the hip?), and virtualization.

    Naturally if you are virtualizing production servers, you need to scale your hardware recommendation up to account for VM overhead.

    What would be very interesting is a study of the bottlenecks. If you are considering a system with certain characteristics (processor/processors, memory, storage/raid etc) and you have X dollars, where is that best spent?
  • by dvdan (1081487) on Thursday March 29, 2007 @09:36AM (#18527353)
    For speed, the newer virtualization tools KVM, Xen, and Virtuozzo are presently substantially ahead of the present incarnation of VMWare. KVM requires the new "hardware virtualization" CPU's from Intel and AMD which must be mentioned here, since they represent a major industry recognition of the value of virtualization. This article seems to be giving people the impression that performance of VMWare Server is indicative of virtualization tools in general, and that all virtualization tools slow down hosted virtual machines dramatically. This is simply false. I know hosting providers running 50 virtual servers on a single dual CPU box with thousands and thousands of users, which would simply not work if all virtualization tools had a 43% hit per instance. Another key matter here is that the author fails to mention (or realize?) VMWare Server is crippleware. VMWare states explicitly not to use VMWare Server for anything other than testing because it does not have the performance or feature set of their full blown ESX and GSX servers. Also, while VMWare may be the oldest and arguably most mature virtualization suite, it is certainly not the fastest.
  • Re:Bogus Test (Score:3, Insightful)

    by afidel (530433) on Thursday March 29, 2007 @09:40AM (#18527407)
    Here's [google.com] a little spreadsheet I created to do a cost/benefit analysis for Vmware ESX. There are some assumptions built in, and it's not yet a full ROI calculator, but it gets most of the big costs. Cell A1 is the number of our "standard" systems to be compared (4GB dual cpu 2003 machines). The DL580 is 4xXeon 7120 with 32GB of ram, local RAID1 on 15k disks, dual HBA's and a dual port addon NIC. The DL585 is 2xOpteron 8220HE with 32 or 64GB of ram (the 580 with 64GB was more expensive than buying two with 32GB!) and the same equipment. The 360 is our standard build currently, dual 5110's with 4GB ram and local RAID1 and an HBA.

    The interesting thing is the breakeven point for VMWare is only 12 servers, way, way below what you can put on two of those boxes. VMotion is the killer feature for me so less than 2 servers is stupid for my situation.
  • Re:Pointless test? (Score:4, Insightful)

    by Albanach (527650) on Thursday March 29, 2007 @09:42AM (#18527433) Homepage
    It's not insane if people want different solutions or even want their own server. With virtualisation, a host can offer multiple php versions. You can avoid all the security problems where one script running as the webserver can read any other file accessible to the web server.

    You can also get better management control of resources, preventing one site from eating up all available resources on the box.

    That's not to say there aren't a million good reasons to use virtual servers in apache, just to point out that virtualising web hosts is not, by definition, a daft idea.
  • Re:Bogus Test (Score:5, Insightful)

    by sammy baby (14909) on Thursday March 29, 2007 @09:44AM (#18527451) Journal
    Yes: it performs much, much better.

    VI3 is actually a suite of products. At the heart is VMware ESX Server [vmware.com], which is actually an operating system in its own right: it runs "on the metal," without having Windows or Linux installed already on the system. It also has a service console operating system which looks suspiciously like a *NIX style operating system, so you can SSH directly to the system, cd into your /vmfs directory and, say, scp disk files over the network. If you wanted to.

    However, as a pretty damn safe rule of thumb, no system is going to run faster on equivalent hardware after being virtualized. In a prior job where I was often asked to provide development/test systems, I got phone calls from a lot of people who were bitten hard by the virtualization bug. Whenever someone brought up any issue having to do with infrastructure, no matter how odd or off the wall, they wanted to push virtualization as a solution. I had to explain to them that if your problem is that a web server is slow, the answer isn't to install VMWare server on it, set up two host operating systems, and say, "There! Now I have two web servers." You'd be surprised how pervasive that sort of thinking is, even among people who should patently know better.

    Another useful guideline: various types of services are impacted differently by being virtualized. Generally, the best candidates for virtualization are ones that spend a lot of time idle. This is actually more common than you might think - people need a server set up for something, can't put it on a pre-existing system for security/compatibility reasons, so they go out and buy a new system which is ten times more powerful than they need. You can put a lot of these kinds of systems on a single, reasonably powerful ESX server. On the other hand, systems that heavily tax available hardware, especially I/O, are usually much harder to deal with.
  • by Joe U (443617) on Thursday March 29, 2007 @10:58AM (#18528407) Homepage Journal
    I do not understand people that make mission-critical IIS-only webapplications, isnt that just stupid?

    No.

    It's no different from focusing on any other major corporate product. ASP.NET isn't going away anytime soon and it's not going to fall over anytime soon.

    It's run by Microsoft. Despite what the ranting morons around here think, Microsoft is not going away, IIS is not an insecure mess and it's actually a really good platform for web applications.

    Now, that being said, use the best tool for the job, otherwise you're just stupid.
  • Re:Well, (Score:5, Insightful)

    by drinkypoo (153816) <martin.espinoza@gmail.com> on Thursday March 29, 2007 @11:02AM (#18528469) Homepage Journal
    To us, the whole point of virtualization is that we have several servers which are mostly idle at all times and completely idle at most times, and for support reasons we're not supposed to be running anything else on the same copy of windows. So we can replace five or six systems with a four-core 1U box with just a few gigabytes of memory, which will recover rack space and reduce power consumption. For anything that's actually heavily loaded, running on the hardware is probably a very good idea.
  • by sheldon (2322) on Thursday March 29, 2007 @11:57AM (#18529311)
    They state in the test that the servers are dual proc servers.

    VMWare Server, the free edition only emulates a single processor environment for your virtualized host.

    VMWare ESX or whatever they are calling the expensive thing today, has the ability to give your virtualized host multiple processors.

    So it's not surprising that it could only handle half the load, it only had half the processors.

    We don't do virtualization for heavy use environments. We do it because different business groups don't want to share servers... that is, they can't agree on maintenance windows, etc.
  • by Quikah (14419) on Thursday March 29, 2007 @02:03PM (#18531213)
    I think everyone is kind of looking at this the wrong way. Sure you get a performance hit, but yu are testing maximum performance. That is a situation where you wouldn't want to virtualize anyway. If the system is running at 100% utilization, then leave it alone. It is more interesting to take your servers running at 20-30% util (if that, how many idle server do YOU have?), and cram them all into a couple of boxes. You most likely WON'T see a perfromance drop because there was so much headroom on the system already. Virtualization struggles at the max utilization case, but then that is not the case that it is really meant for.

Stellar rays prove fibbing never pays. Embezzlement is another matter.

Working...