Forgot your password?
typodupeerror

Generator Delays May Slow Data Center Projects 257

Posted by ScuttleMonkey
from the so-that's-why-mmos-can't-expand-to-meet-needs dept.
miller60 writes "The data center building boom is causing backlogs for new generator orders, with some companies reporting delivery delays of up to a year for new 2,000kw units, which are the current standard for mission-critical facilities. Generator availability is 'the No. 1 thing that will drive your construction schedules,' according to Equinix, which is building centers in three major markets. 'This will be a big issue for the next wave of data center builds,' says another industry executive. Used generators and smaller units tend to be more available than the 2 megawatt units, but companies targeting the enterprise sector may be wary of relaying on used units or smaller generators than those powering competing facilities."
This discussion has been archived. No new comments can be posted.

Generator Delays May Slow Data Center Projects

Comments Filter:
  • by hurting now (967633) on Wednesday October 25, 2006 @01:06PM (#16580696) Homepage Journal
    You need to remember, its not just the computers & servers. Its the lights, the heating/AC, all other power needs. We have a generator at my work and it has to power EVERYTHING. 100 desktops 34 thin clients, 19 laptops, 34 servers, the A/C for the data center, the elevator, the emergency lighting, fire detection system, phones... you get the idea. We have a 30 KVA battery backup for the data center in case the generator doesn't kick on the right way.
  • not on my project (Score:4, Informative)

    by superstick58 (809423) on Wednesday October 25, 2006 @01:09PM (#16580752)
    I didn't see any delays on a project I am working on. We have 4 Gens in the 2MW range plus 3 gas/diesel gens in the 30MW range. Of course, this isn't for a datacenter, but I wonder if the generators would be the same.
  • Re:2 MEGAwatts?!?! (Score:5, Informative)

    by Volante3192 (953645) on Wednesday October 25, 2006 @01:12PM (#16580782)
    Well, converting an existing engine gives you the same, if not more, concerns as getting a used generator. As TFA mentioned, there's no problem finding used 2MW generators, but it's the stigma of using used equipment period.

    Add to that the changes and parts necessary to change engines geared to creating propulsion to engines geared to creating electricity.

    I doubt any company who considers their data center a key component of thier infrastructure to risk their backup solution on an untested refurbished generator.
  • Re:not on my project (Score:3, Informative)

    by AKAImBatman (238306) * <akaimbatman@[ ]il.com ['gma' in gap]> on Wednesday October 25, 2006 @01:12PM (#16580796) Homepage Journal
    We have 4 Gens in the 2MW range plus 3 gas/diesel gens in the 30MW range.

    What are the 4MW units if not diesel? If you're using Gas Turbines, that may explain why you didn't have the same supply problems.
  • Re:2 MEGAwatts?!?! (Score:5, Informative)

    by jeffmeden (135043) on Wednesday October 25, 2006 @01:22PM (#16580990) Homepage Journal
    On your comment about data center size: Consolidation is the buzzword of the decade. 'Big boy' data centers start at 2MW and get BIGGER. It's not uncommon for a main site facility to be closer to 4MW. These are usually fed from the grid at the 14kV level, often from dual substations. Yes it's a lot of CPUs and supporting equipment, but thats the way we like em!

    Your question on two generators vs. one: Redundancy does NOT come from two half sized units doing a single job, it comes from two FULL sized units each doing half the job. Having two half sized generator units means losing one will cause failure. With a 10,000HR MTBF per unit (rough number used for demonstration) you get a system MTBF of 5,000HR, NOT what I would call redundant! You would need at least 3/2 redundancy (3 generators doing the work of 2) to have a reasonably reliable replacement for a single large unit.
  • Re:2 MEGAwatts?!?! (Score:3, Informative)

    by DigiShaman (671371) on Wednesday October 25, 2006 @01:35PM (#16581218) Homepage
    I doubt any company who considers their data center a key component of thier infrastructure to risk their backup solution on an untested refurbished generator

    New or used, it doesn't matter. Your SUPPOSED to testing the generator at least once a week. Besides, it's not good to have aged fuel in the tank. You end up with all sorts of fuel-line and injector clogging due to the fact fuel will take on properties of varnish.

    Point is, it does't matter if it's used. If it does the job and can be verified to do so once a week, you're ok.
  • Re:Easy, just buy... (Score:5, Informative)

    by Detritus (11846) on Wednesday October 25, 2006 @01:43PM (#16581366) Homepage
    Then you have to keep them phased together. That can be complicated and error-prone. When you're operating on diesel power and lose a generator, the load gets dumped on the remaining generators, which can cause large frequency errors and brownouts. To put a spare generator online, you have to match frequency and phase with a system that is under severe stress. It can get ugly.
  • Re:2 MEGAwatts?!?! (Score:2, Informative)

    by Critical Facilities (850111) on Wednesday October 25, 2006 @01:47PM (#16581440) Homepage
    Yes 2 Megawatts. I work running the engineering department for a major financial provider and our campus has two 2 Megawatt generators dedicated to our datacenter (as well as 2 more for the rest of the campus). We actually have most of the infrastructure in place for a third 2 Megawatt generator as we develop the need for power.

    You'd be surprised at how much the average load in KW/sq ft has increased in the last several years for a datacenter. Moore's law is FIRMLY in place and it's getting harder and harder to provide adequate, clean, redundant power in critical environments. Server manufacturers are really upping the ante on power consumption which in turn ups the ante on cooling the space(s).

    As for your question "wouldn't it make sense to get two smaller units?" , the short answer is no. Mainly, you'd have to have more electrical components (switchgear, breakers, transfer switches, etc) in order to have more than one, plus you've now got another generator to maintain and you're going to need the physical space for it (and all the gear associated with it which is quite a lot). Add to that all the safety equipment you need (FM-200 fire supression, CO2 fire suppresion, Building Automation monitoring, fuel level/leakage detection) and you get really expensive really quickly.
  • Re:2 MEGAwatts?!?! (Score:3, Informative)

    by Dun Malg (230075) on Wednesday October 25, 2006 @01:55PM (#16581584) Homepage
    So you are saying that having two smaller generators instead of one large one doubles the risk of power failure?
    Doubles the risk of a show-stopper failure in the backup system, yes. Twin engine aircraft are required to be able to fly with one engine out for this very reason. A crude way to visualize it is that the MTBF rates are like unto the chance of rolling a Critical Failure in [your favorite RPG]. One generator rolls the dice every (x) amount of time, and TWO generators is essentially rolling the dice TWICE every (x) amount of time. Odds of rolling a critical failure statistically double.
  • Re:DC power? (Score:2, Informative)

    by RxScram (948658) on Wednesday October 25, 2006 @01:56PM (#16581600)
    In a former life, I worked on large (800KVA) uninteruptible power supplies as a field service engineer. Normally, for highly mission critical data centers, as described in the article, the data center is powered by 2 different, independent power grids (if possible.) For example, there would be one feed from power company A, and another feed from power company B. These would be connected by using an automatic transfer switch (ATS) which would immediately switch to the other power feed if the main failed. The output of this ATS would then be connected to another transfer switch which was connected to the emergency generator, which starts automatically in the event of a power failure.

    On the downstream side of this second ATS (the one that switched between the redundant power company feeds and the generator feed), there would be the redundant UPS network, which would in turn feed a different type of transfer switch before heading to the actual critical load in the data center. Each of these multiple redundant UPS's were provided backup power with large strings of batteries. A typical 800KVA UPS would have 3-4 strings of 40 batteries each. Each of these batteries would cost around $100-250, and weigh about 100 lbs. The data centers I worked on, which were nowhere near as large as some of the MS or Google data centers described in the article, had between 4-16 independent, redundant UPS's.

    How long would this huge amount of batteries keep the UPS up at full load in the event of a power outage?

    About 15 minutes, if you were lucky and had batteries at the top end of their charge cycle.

    The whole purpose to having the UPS redundancy and massive battery backup is to give the emergency generators enough time to start and come online.

    When I was working at MGE, a typical figure we were told was that a minute of downtime for some of these data centers can cost the datacenter well over a million dollars in lost revenue, and immediate termination of the CIO (or whatever equivalent) of the company. I'm not sure how accurate this was, but it seems reasonable considering how much money goes through some of these places in a day.
  • Re:2 MEGAwatts?!?! (Score:3, Informative)

    by Politburo (640618) on Wednesday October 25, 2006 @03:00PM (#16582694)
    You'd have to look into the air pollution regulations. Depending on your location, there are fairly strict guidelines for generator engines. Staring in March, in NJ your generator cannot emit more than 1.5 g/bhp-hr of NOx no matter when you installed it (0.90 g/bhp-hr for new units). The current EPA standard (1997) for new train engines ranges from 5.5-14.0 g/bhp-hr (there is a stricter standard in the works). The current EPA standard (2004) for heavy-duty diesel vehicles is 2.4 g/bhp-hr. An old train or truck engine isn't going to cut it.
  • by Critical Facilities (850111) on Wednesday October 25, 2006 @03:10PM (#16582838) Homepage
    "Modern Generators" as you call them do not happily sync together under load. If you have PLC Controlled Switchgear associated with them, then you will be able to sync them, and even then they'll never be perfect. If one generator lags and becomes a "slave" it also becomes a load to the "master" generator, thus reducing your available power to the critical load. It's definitely not as simple as just dropping a few generators in a row and wiring them to the same buss, I assure you.

    Also, a 1MW generator is pretty large, as is the switchgear and any fuel storage tanks associated with it. As far as the UPS's, true, they will take care of brownouts and frequency errors, but you want the power coming in (either utility or generator) to be as clean as possible so that you're not beating the shit out of your batteries every few minutes to take care of spikes. Batteries do have a shelf life and they're not cheap.
  • by Above (100351) on Wednesday October 25, 2006 @03:16PM (#16582916)
    I see a lot of posts that 2MW is a lot of power. Perhaps.

    There are many facilities out there with 10-30 2MW generators. I personally worked on a facility that had 18. 12 were for "critical load", that is, things fed via UPS's. This included all of the servers and networking equipment, the NOC, emergency lighting, fire supression systems and soforth. The other 6 were for "emergency load", these were things that did not need UPS protection (they could go down), just not for long. Virtually all of this load was air conditioning. So when the power went out the A/C's went off and came back 30-60 second later on generator, while the servers and all stayed up the same time.

    While big, it's far from the largest facility out there.

    Now, why would you buy one generator? Well, many buildings use bus variations (the whole N+1, or N+2 thing). So you build your data center for all 18 generators (as above), but install 6 and 3, or half the capacity. You now wait for the building to fill to 25-30%, and then start adding more generators, one at a time. However, they now have to be matched to the other generators.

    Now, why are data center generators special? Well, to switch from one AC source to another the two loads must be approximately in phase (there is some tolerance, but it's small). So in order to be able to switch between generators, switch from UPS to generator, and all that other stuff you need additional circuitry to keep the generator just so. While the engine block and generator were the same as say a diesel locomotive, there were some additional sensors, lots of additional computer control, and some additional quick start features.

    Most sites want their backup generators to be "ready to load" in 15-30 seconds. Indeed, several manufacturers make 10 second ready to load units. However that requires things like engine oil and coolant heaters that basically keep the entire motor warm 24x7x365. These are not found on industrial generators or locomotives.

    So, they really are special, high tech tight tolerance units designed to work in a system. When you connect to WoW or your other large game you're probably one one of a thousand servers run by that company, who is one of but hundreds of companies in the larger colo facilities. It's not uncommon to find 50,000 servers under a single roof. So you need lots, and lots, of 2MW generators.
  • Well, yes and no (Score:4, Informative)

    by Flying pig (925874) on Wednesday October 25, 2006 @03:57PM (#16583590)
    Nobody will read this...

    Your product is interesting but there is a difference. The reason the cost per KW of engines over about 500KW starts to rise is that they are designed for very long life. Unlike gas engines, where when you get outside the cylinder size range of about 50-500cc things start to go downhill, Diesels scale to enormous cylinder sizes but, because the mean piston speed needs to be constant, power goes up as the 2/3 root of cylinder volume. Mass scales more closely to volume, so mass rises faster than output. However, the benefit is that the wear life of a larger cylinder is much greater, because it can tolerate larger amounts of linear wear before blowby becomes excessive.

    The implications are that while the cost per installed KW of your multi-engine plant is about the same as that of a single 2MW unit (because of the additional switching and control gear) it will not have the same service life and its lifetime cost per KW is higher.

    Correctly and honestly, you describe your generators as backup. But larger units in the 2MW and over range can be used as primary generators. That's the difference. As somebody else has observed elsewhere, large units can be remote controlled by electricity utilities for handling load peaks. This means that their asset utilisation can be much higher than standby generators in well designed stationary applications. The lifetime cost per KWH of a backup generator can be very high because its first cost is amortised over low running hours.

    Fact is, 2MW isn't a big Diesel. It's portable power (as used in trains and boats.) That's why supply and demand is likely to vary according to major events like earthquakes and wars.

  • At my last job.. (Score:2, Informative)

    by illuminix (456294) on Wednesday October 25, 2006 @04:38PM (#16584216) Homepage
    Where I used to work, they had 6 diesel locomotive engines, all in a row in a big room. When I went on the tour, they told us we could only stay in there for a minute, because of the possibility of them firing up at any time (the sound would be deafening). Each of the engines sat on an isolated "pad" which reduced vibrations. They told us that was necessary to prevent the harmonics for damaging the building. It was really a beautiful thing :)

    In the summer, they would sometimes run them during peak periods. I was told that they generated enough power to run the datacenter (which is one of the largest in the midwest), plus sell enough power back onto the grid to earn a sizeable kickback.

    Technology is cool.

Nothing is faster than the speed of light ... To prove this to yourself, try opening the refrigerator door before the light comes on.

Working...