Efficient Supercomputing with Green Destiny

gManZboy writes: "Is it an oxymoron to have an efficient supercomputer? Wu-Chun Feng (Los Alamos National Laboratory) doesn't believe so - Green Destiny and its children are Transmeta-based supercomputers that Wu thinks are fast enough, at a fraction of the heat/energy/cost, according to ACM Queue." 240 processors running under 5.2kW (or less!) is nothing to sneeze at. The article offers up this question: might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

Holy crap!

I knew that sword was beefy, but that's insane!

Indeed...

Most definitely nothing to sneeze at. I have to ask, though: how long ago was it insane for a supercomputer to put out as much heat as the average enthusiast PC puts out today?

Perhaps....

How much of a footprint and weight they take up as a metric to consider? ;)

Old age question for a new generation

The MHZ war has been going on for soooo long that everyone just excepted that faster MHZ related to faster machines. Well, 64Bit computers are placing chip manufactures in a position where they have to market on a platform that declares that MHZ doesn't really matter.

I think the question is a bit naive though as everyone knows a hundred software tools to rate performance of CPUs rather than just relying on MHZ.

Nick Powers

Re:Old age question for a new generation

Just look at cars - time was the only thing many people would look at is cubic inches or horsepower. Now most people who buy a car are more concerned with other features - passenger comfort, style, efficiency. I would guess this is a shift from car-oriented people buying cars to everyone buying cars as they became more of a necessity.

Computer manufacturers are only just starting to see this, making smaller, quieter, cooler-running machines. Hopefully they'll continue to look at what their customers actually need rather than simply putting out chips with higher clock speeds.

Imagine...

a (kinder, gentler) Beowulf cluster of these!

Speaking of which, why hasn't anyone made an OpenMosix cluster-in-a-box yet?

WTF?

Why bother? If you have to sacrifice computational power for energy efficiency, then what is the point of having a supercomputer? Isn't compute power the whole purpose of having a supercomputer?

Re:WTF?

No, supercomputers that can do a lot of image processing cannot waste power simply because it might be available.

Modest supercomputers are used in the military on airframes. Power consumption is important for at least two reasons. First is the wattage and power draw. Second, and more subtle, it that the cooling requirements while flying at high altitude become more important than simple fan noise. Pentiums burn up no matter what you do. PowerPCs@10Watts with conduction cooling will survive.

Would native VLIW be better?

For this specific application, scientific supercomputing with a blade architecture, would native VLIW offer any performance benefits?

If it would only be a few percent, it wouldn't be worth it. Transmeta has picked the game they want to play, and it would be a big deal at this point to engineer a special version of their chips that make it possible to run native VLIW code.

I'm guessing that typical scientific processing involves a lot of loops that run many iterations, which is the ideal situation for the code morphing engine; so I'll go out on a limb and predict that it is not worth it to make a special version of the chip.

steveha

Does it really save that much power?

I was talking to a friend the other day about a bunch of lab computers that my school is getting rid of - a bunch of old Pentium MMX's. He suggested turning them into a cluster. But after thinking about it, I realized that the group of about 10 old computers we had would consume more power - and would likely be considerably slower than a single one of the 2.4Ghz Dell's that replaced them. "What's the point?" I said.

Applying that here, the little VIA chips run at roughly the speed of a Celeron 500 or so, I'd say something like an AMD Athlon 3GHz would be just about as fast as about 6 of the VIA chips. So you are still saving some power, but as not as much as it would seem as first, as you need many low power chips to equal the speed of one faster chip. Not to mention power consumed by having more motherboards, network cards, switches, and other associated hardware.

Something to really look at is the cluster of G5's. The G5 chips use a lot less power than their x86 counterparts. I bet that cluster of G5's is probably right up there in terms of processing power per watt as this VIA super computer. And it's way more cool to boot.

Re:Does it really save that much power?

You're comparing apples to oranges, not to mention that your info's a little off...

1) A Nehemiah core C3 runs really close to the same performance of a comparably clocked Celeron, with the same general power consumption of a Samuel2 core (For those that don't know, part of how VIA's chip originally got it's low power is that the FPU was underclocked by a factor of 1/2). It's a nice chip overall, but it's not really intended (nor are they USING it that way) for scientific or gaming applications even though you can use it for that. The C3's winning usages is in something like a media PC, workgroup servers, and embedded systems where you need low power consumption, relatively low cost, and relatively high performance compared to other x86 embedded solutions.

2) The Crusoe and similar chips are very fast executing VLIW CPUs (very much like the Itanium...) that have code morphing that translates x86-32 instructions into comparable sets of instructions for the VLIW CPU- in fact it's very good at doing this sort of thing. The reason it's less desirable with a desktop or gaming application is that you're exceeding the VLIW code cache regularly, meaning you have to keep recompiling the x86 instructions into the native VLIW ones. For a scientific application, the same task gets executed time and time again and usually ends up with most, if not all, of the code in the pre-morphed code cache. At that point, you're now in the high-performance domain with very little power consumption. The Crusoe in this application would consume less power than the G5 and run just as fast. (Check the article that you're commenting on...)

Do some thinking outside of the box here, what's good or great on a desktop machine isn't always the optimal choice for supercomputing clusters or HA clusters. Depends on a bunch of factors, including what you're going to be running on the systems in question and what kind of environmental conditions you're going to be facing.

Hmm...

While the points that the author makes are true about the "frugal consumer", those aspects are not applicable to supercomputing.

Overall performance is much more important than efficiency. While efficiency is commendable at all computing levels, if efficiency is a very important aspect, then a supercomputer is probably not for you.

Do the math

This is very, very cool. For one thing, a bottleneck in supercomputers is in most cases the network. In this regard, dropping some per/node performance might not affect the overall performance for applications that need intensive interprocess communication.

The other point is: how expensive it is to support a cluster ? Not only the energy consumption, but also the infraestructure. It is pretty darn difficult to keep a thousand processors cold. You may need a special building, special power supply for it, etc.

A final point: as far as I know, the rule of thumb is that the floating point performance with these energy efficient processors is of the same order of magnitude as regular processor, may be a factor 2 difference.

You do the math ... :-)

Bigger clusters

If your supercomputer cluster nodes are cheaper/more energy efficient, then given a fixed budget, your supercomputer can be bigger!

Laptop/Desktop/Handheld Supercomputers?

This makes me wonder if such a configuration might find its way into ordinary extreme performance desktop/laptop computers.

Especially with the new wave of Media Center based PCs...small small machines that are very powerful....is THIS the future of servers? Perhaps in a few years my web pages will all be served up from something like a handheld PC, with several processors and always-on WiFi? The possibilities are endless, but I see this DEFINITELY making it into laptops of some creed...those ultra-high-performance ones that nobody seems to buy.

Vivan los pocos!

supercomputers vs man's only finite resource

Why are supercomputers primarily benchmarked by their speed? The answer comes when you consider that almost all labour-saving devices are measured in the work they perform in a given period of time.

Time is the only truly finite resource from a human perspective. As technology has progressed, distances have been conquered, vast energies harnessed, but old Father Time is still inescapable.

As a result, we place great value on just how much time is taken to accomplish anything.

Nano-ITX

with the centaur C5P processor core. Draws about 8W for the chip @ 1Ghz. Lets assume 12W total for network boot.

[ see image here: peertech.org/hardware/viarng/image/nano-itx-c5p.jp g [peertech.org] ]

With 5,200 Watts for Green Destiny, you could use 433 boards these boards for the same power consumption.

The on chip AES is clocked at 12.5Gbps, Entropy at 10Mbps (whitened). Thus you would have

422Ghz of C5 processor power
5.412TB/s of AES (yes, terabytes)
4.22Gbps of true random number generation.

Yeah, these are really rough estimates, but that is a long of bang for your kilowatt buck no matter how you slice it.

With a cutting edge P4 approaching 100W the efficiency of these less powerful but fully capable systems will become increasingly attractive.

I would not be surprised to find bleeding edge processors relegated to gamers and workstations as most computing tasks start migrating towards small, silent, low power systems that simply *work* without eating up desk space, filling a room with fan noise and driving the electricity bill higher with continuous 100's of W draw.

more important metrics that just speed?

might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

Yes, people often consider flops/watt to operate, and flops/dollar to buy.

Speed alone means nothing. All these atoms in my apartment can do billions of operations per second, but they can't even play mp3s.

well hey

remember when computers were big enough to be in warehouses? and 10 years ago, people theorized that computers would be small enough to fit in your watch or hand? and that was just theory and considered fiction.

now we have palm pilots and watches that can store data (see the usb wrist watch)

so, really, a supercomputer that doesnt use that much energy isnt impossible.
anything's possible, one just has to break through the set barriers technology has made. if no one did that, we still would be sitting around in caves.

Times change

Lottsa years ago I used to maintain a CDC 7600, not only did it need full refrigeration, but it's original design spec was for an MTBF of 15 hours! The designers reckoned that it was so fast that the biggest job imaginable could be run in that time. Of course it did better than that in the end, but it was a bugger of a job to fix, and the backplane was 6 inches deep in twisted pair wires. Just imagine making wiring changes.

from the so-obvious-it-hurts dept.

The article offers up this question: might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

Um, yes?

Do the comparison with VT X instead of ASCII Q

If you do the math with X (10,280 instead of 13,880 performance, 1000sq instead of 21,000sw, and 800kw instead of 3,000kw) you get a 337 fold increase in performance per square foot, rather than 65, and an 832 fold increase in performance per Watt, rather than 300 fold, vs the Cray.

Of course I dunno the numbers for the Transmeta solution yet!

NOP like there's no tomorrow!

. . . might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

No. I have a rock that can sit and do nothing, consuming considerably less than even 5.2kW. You can talk efficiency and bang-for-buck all you like, but if you don't benchmark faster than (roughly) 100 common desktop machines, you don't get to call yourself a supercomputer.

Do companies care?

The article offers up this question: might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

Yeah. Does the supercomputer do what the customer needed it to do? Nobody in the world lays down money for a "supercomputer" these days so that they can be the fastest kid on the block ... or at least they shouldn't. Ostensibly, there are massive amounts of computing work that they need done, and they need something that can do it in a reasonable amount of time. Beyond that ... was it worth their money? If the answer to the last two questions was yes, then it was a success. Supercomputers are a tool; a very large tool, but a tool nonetheless.

Imagine reading the beowulf mailing list! :-)

Anyone interested in this sort of thing should check out the Beowulf mailing list - go to www.beowulf.org, and read through the (recent) archives. There's been some talk lately on different metrics.

Memory Speed

Its not CPU speed that is important in supercomputer/clusters it is the speed at which you can get data from one node to esp memory access. If you havea 512 node system and node 3 needs a copy of node 40's memory it has to copy it over.

If its even just 512Mb of Gigabit ethernet and assuming 100% performace it would still take 5 seconds which is many orders of magniture. Just look at SGI machines which use NUMA and their Cray-Linux are 3.2 TeraBytes (bytes not bits). Now thats how you want to shift data

Rus

Efficiency

Is it an oxymoron to have an efficient supercomputer?

I thought heat was the real poison of "ultimate" computing...
So it seems likely computers will move towards those limits and become 'greener' with respect to how much energy they use...

To do otherwise would be counterproductive in terms of both efficiency and ecology.
That's needed given how much energy the US is using [cornell.edu]

Not a bad thing - but I wonder when green will move towards a technology that means less polluting in terms of hardware that gets trashed every year.

Being green is important

Especially when simulating nuclear weapons.

-Shane

depends on the TASK...

It's still valuable to have one or a few really friggin' fast processors versus a whole lot of smaller processors if you're running tasks that can't easily be subdivided. This is why people are still buying single processor PCs rather than multiprocessor boxen. If you're buying the setup for a specific purpose and multiple slower CPUs will do the job for you, then that's great; but you'll get more flexibility with speedy processors.

Less heating=Denser packing

If one can pack the processors more densely, it would cut down on the wiring etc, or allow much shorter paths between nodes (better still, one might be able to stuff many processors on the same board or something), thereby increasing bandwidths (when you try to increase bus speed, path length and related current leakages etc do pose problems). This in turn means computations that require more 'random' communication between nodes can speed up. I suppose that's definitely worth pursuing for the more fine-grain computation where communication bandwitdh is the bottleneck.

Green Destiny?

Wu-Chun Feng (Los Alamos National Laboratory) doesn't believe so - Green Destiny and its children are Transmeta-based supercomputers that Wu thinks are fast enough, at a fraction of the heat/energy/cost, according to ACM Queue.

Yes, yes, those numbers are impressive but can it be used to destroy other weapons and conquer the Chinese underworld in the hands of a rebellious Manchurian girl? (Reference: Crouching Tiger Hidden Dragon)

Why does a supercomputer need x86 compatibility?

That is the only advantage of using a Transmeta CPU. Wouldn't it be more efficient to just use a regular VLIW CPU without all the x86 code morphing stuff?

Steam powered PC

I wonder if it's actually possible to reuse the heat generated by your average overclocked processor.

Maybe it would be possible to convert the heat from the processor into more electricity somewhere else. That would be cool.

The short answer?

No.

Faster, ever faster!

Full Formatted Text

Sorry, no Tables and no Pictures

Making a case for Efficient Supercomputing
From Power [acmqueue.com]
Vol. 1, No. 7 - October 2003
by Wu-Chun Feng, Los Alamos National Laboratory It's time for the computing community to use alternative metrics for evaluating performance.Motivation

A supercomputer evokes images of big iron and speed; it is the Formula 1 racecar of computing. As we venture forth into the new millennium, however, I argue that efficiency, reliability, and availability will become the dominant issues by the end of this decade, not only for supercomputing, but also for computing in general.

Over the past few decades, the supercomputing industry has focused on and continues to focus on performance in terms of speed and horsepower, as evidenced by the annual Gordon Bell Awards for performance at Supercomputing (SC). Such a view is akin to deciding to purchase an automobile based primarily on its top speed and horsepower. Although this narrow view is useful in the context of achieving performance at any cost, it is not necessarily the view that one should use to purchase a vehicle. The frugal consumer might consider fuel efficiency, reliability, and acquisition cost. Translation: Buy a Honda Civic, not a Formula 1 racecar. The outdoor adventurer would likely consider off-road prowess (or off-road efficiency). Translation: Buy a Ford Explorer sport-utility vehicle, not a Formula 1 racecar. Correspondingly, I believe that the supercomputing (or more generally, computing) community ought to have alternative metrics to evaluate supercomputersspecifically metrics that relate to efficiency, reliability, and availability, such as the total cost of ownership (TCO), performance/power ratio, performance/space ratio, failure rate, and uptime.

Motivation

In 1991, a Cray C90 vector supercomputer occupied about 600 square feet (sf) and required 500 kilowatts (kW) of power. The ASCI Q supercomputer at Los Alamos National Laboratory will ultimately occupy more than 21,000 sf and require 3,000 kW. Although the performance between these two systems has increased by nearly a factor of 2,000, the performance per watt has increased only 300-fold, and the performance per square foot has increased by a paltry factor of 65. This latter number implies that supercomputers are making less efficient use of the space that they occupy, which often results in the design and construction of new machine rooms, as shown in figure 1, and in some cases, requires the construction of entirely new buildings. The primary reason for this less efficient use of space is the exponentially increasing power requirements of compute nodes, a phenomenon I refer to as Moore's law for power consumption (see figure 2)that is, the power consumption of compute nodes doubles every 18 months. This is a corollary to Moore's law, which states that the number of transistors per square inch on a processor doubles every 18 months [1]. When nodes consume and dissipate more power, they must be spaced out and aggressively cooled.

Figure 1

Without the exotic housing facilities in figure 1, traditional (inefficient) supercomputers would be so unreliable (due to overheating) that they would never be available for use by the application scientist. In fact, unpublished empirical data from two leading vendors corroborates that the failure rate of a compute node doubles with every 10-degree C (18-degree F) increase in temperature, as per Arrenhius' equation when applied to microelectronics; and temperature is proportional to power consumption.

We can then extend this argument to the more general computing community. For example, for e-businesses such as Amazon.com that use multiple compute systems to process online orders, the cost of downtime resulting from the unreliability and unavailability of computer systems can be astronomical, as shown in table 1millions of dollars per hour for brokerages an

Other Metrics...

yes, cost...

New trend in computing. Vector processing

Has anyone else noticed that vector processing is gaining momentum ? Some array processing links .

• 25 gigaflop CS301 clearspeed vector microprocessor [newscientist.com]
• clearspeed [clearspeed.com]
• General Purpose Computation Using Graphics Hardware -Gpgu.org [gpgpu.org]
• Is it behind IBM's cell processor ??? hmmmm [com.com]
• IBM and Texas University's Trips 1 trillion ips vector processor [ibm.com]

Speed is not always best

Sure, if one has the budget to build a better supercomputer, like the NCSA, then all one would be interested in is speed. That's great... unless you can't afford it. I am sure that there are many places, like community colleges, small-scale research labs, and other considerably lower-budget places that would greatly appreciate having a supercomputer brought down to a possibly acceptable level. Of course these places would love a top-notch computer, just as you lust at the Ferrari dealership... on your way to the Honda dealership to see about that cute little Civic. Same thing for the smaller institutions. Who cares that your computer is insanely powerful when you can't afford to turn it on? I think that's the aim of the whole efficient supercomputer idea. An OK-speed supercomputer that an institution can afford is better than a powerhouse that it can't.

This makes little sense to me

It would seem to make more sense to shut down and start up individual nodes for power saving. A supercomputer has relatively little down time and most of the time jobs come in large batches, you can shut down or heat up additional nodes as needed. It should be relatively easy to implement this kind of power saving inside single "computers" with many processors, and of course absolutely trivial to do it with a cluster. Especially if you use WoL NICs.

Other metrics

"might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?"

Sure. How fast Quake can run on a Beowulf cluster of them.

To a lot of people, nothing else matters. Me, I like processing power for its own sake.

Efficiency?

No-one seems to have noticed that SGI's R16k CPU uses a grand total of 12W power, and does 1.4 GFLOPs. That's 512 of them for your budget of 5kw. And that's single-system-image, and it'll all fit in 4-5 full racks.

[yes I work for SGI, but I'm not in marketing.. since marketing never DO anything useful I thought I'd better..]

"Fast enough" != "Supercomputer"

My desktop machine is faster than a Cray 1 [ed-thelen.org], and it'll never be labelled "Supercomputer" by any rational being.

Unless their architecture actually hits the Top Ten [top500.org], I'm not going to be impressed that it's overcoming its handicap. Unless you're running a Special Olympics [specialolympics.org] for computers and "everyone's a winner."

Green Destiny

Li Mu Bai wants his sword back

Finally!

This is what Transmeta needs to hype. Their processors are slow, and most people who have laptops plug them into walls 90% of the time, so battery life isn't all that important.

OTOH, a power-efficient, low-heat cluster with a cooling system that doesn't cost more than the machines that make the cluster is nothing to laugh at. Buy a transmeta-based cluster and you don't need a special contract with the electricity company! You don't need five-feet-thick walls for sound insulation! You don't need to pipe liquid nitrogen to your supercomputer!

It's what we lose sight of

Enough computing power is the right amount to do the job require.

Yes it's a statement of the obvious but I'm amongst what I suspect is the majority that loves the idea of having a multi 64 bit CPU as soon as I have spare cash.

I don't need that amount of power. The reality is my 700 Duron is still adequate though not totally adequate.

StrongARM

If MIPS/Watt is the focus, why not use Intel's StrongARM, XScale or other ARM [arm.com] based cores rather than Transmeta's stuff. Afterall, ARM was designed specifically with the MIPS/Watt ratio as objective, starting a whole new architecture from scratch. Whereas Transmeta has focussed on effectient x86 "emulation".

--
Real computer scientists despise the idea of actual hardware.
Hardware has limitations, software doesn't.
It's a real shame that Turing machines are so poor at I/O.

Power Efficiency

This is a metric that could be used. How much performance can you get per watt?

For supercomputing, I would imagine that something like SPECfp/watt or SPECrate/watt would be a decent metric.

If your limitation is a finite power budget, then you pick the most highest perf/watt CPUs.

P4 3.2 EE = 18.44 SPECfp/Watt (80 watts)
Crusoe = ?? No performance numbers published, but I'll bet you it's lower

Building larger caches (which can be made low power) is a good way to acheive high power/performance efficiency. Crippling your performance with a VLIW seems like a bad choice. You can voltage scale down a Pentium-M to the power levels of a Crusoe and easily get 2-3x the performance.

Duh

The article offers up this question: might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

Not to sound flippant, but...duh. Okay, I know I sound flippant. But seriously, why has it taken so long to realize that processing speed is not of the utmost importance? It's like saying one car is better than another because has a top speed of 180MPH and the other 174MPH, ignore that the "slower" car gets 30% better mileage. There's such a thing as total cost of ownership.

Now if only Slashdotters would realize that this applies to home systems and not just supercomputers.

That is all fine and good, until....

Yeah, just wait until Li Mu Bai finds out, he is going to be pissed.

Embedded systems have always...

The article offers up this question: might there be other metrics that might be important to supercomputing, rather than relying solely on processing speed?

Embedded systems (and I'm not just talking about microcontrollers in your phones or microwaves - I'm talking about 100s of processors connected together in VME cages and the like - see Mercury, CSPI, Sky for examples) have always had the metrics FLOPS/W (FLOPS per Watt) and FLOPS/m^3 (FLOPS per volume) metrics. These were critical measurements because applications required certain performance and the machines themselves had to meet size/weight requirements depending on where they were being deployed. Jamming many processors in the space of a microwave oven to meet performance requirements (like 64+ processors), being less than a certain weight, having power consumption constraints, and requiring high performance without melting down because of the heat has always been an issue in certain sectors.

In the past, the embedded systems were typically special purpose - ran "special" OSs and were basically big set-top boxes that did only the one thing they were programmed to do. However, a few years back (like 5+), companies like CSPI started doing things like running Linux (or a realtime Linux variant) on their nodes instead of VxWorks and such, turning the box into a general purpose machine. I guess it just takes a while for some things to get enough attention to where someone would post it on /.

They modified the code morphing software?

Although the Transmeta processor is significantly more reliable than a conventional mobile processor, its Achilles' heel is its floating-point performance. Consequently, we modified the CMS to create a "high-performance CMS" that improves floating-point performance by nearly 50 percent and ultimately matches the performance of the conventional mobile processor on a clock-cycle-by-clock-cycle basis.

That's not exactly trivial. Does anyone know wnything more about this? What are the trade-offs?
And, will this code will be made available to the general public? I imagine there are Transmeta owners that would also like to double their floating point performance.

Jon Acheson

Re:Green Destiny

Robert Cringely pointed out [pbs.org] the benefits of this tradeoff (pure speed vs. low heat/hihg maintainability), pointing to Google's use of Pentium III-s for their server farms.

Re:Green Destiny

Mod parent DOWN -- was copied from another post!!

Re:I can think of one

if the inside contains a computer that's half the speed of competing x86 boxes

Let me get this straight, you're comparing overall computing performance of a chip based solely on its clock speed? You must be new around here.

Re:Green Destiny

Oh bullshit. This is a total troll, are you kidding me?

5.2kW cannot be sucked out of "a normal building power strip." And you are sure as heck going to notice 5.2kW of heat, and the regular everyday HVAC is most definitely not all it requires. "Uncooled ordinary room" my ass.

Re:Green Destiny

thanks for noticing and giving me the credit. the original post was indeed mine. this pig copied it.

Re:I can think of one

Have you ever tried picking one of those things up? I have. I worked on the G5 cluster. Those SOBs were heavy. Nice to look at, but they suck to bring to LAN parties.

Re:green destiny?

Green Destiny sounds like a new strain of high-powered, genetically engineered weed.

I could just see the DEA raiding Los Alamos after getting reports of computer scientists blowing entire afternoons on "their smoking Green Destiny"