Supercomputer to Hit 1.6 Petaflops With 16,000 Cell Chips

tygerstripes writes, "IBM has announced that they are gearing up to build the world's fastest supercomputer, more than four times faster than the reigning champ, IBM's BlueGene/L. Nicknamed 'Roadrunner,' the new machine will be a hybrid of off-the-shelf CPUs and Cell chips designed for the PS3. Roadrunner is to be installed at Los Alamos National Laboratory, occupying 1,100 square metres of floorspace (that's a square about 110 feet on a side). According to the BBC: 'The computer will contain 16,000 standard processors working alongside 16,000 Cell processors... each Cell is capable of 256 billion calculations per second.'"

Now...

[ronadams]

OS/2 compiles your homemade C code faster than you've ever seen before!

Re:

[tha_mink]

Wow, Imagine a ... ah nevermind.

Re:

[cashman73]

Imagine how well this thing could play Duke Nukem Forever ?!?! When does that game come out? ;-)

Re:

[Pharmboy]

Imagine how well this thing could play Duke Nukem Forever ?!?! When does that game come out? ;-)

Actually, this will be the "minimum requirements" to play DMF by the time it comes out. Obviously you need more RAM if you want to experience the advanced features...

Whew, right under the wire!

[BadAnalogyGuy]

Just in time for the Vista RC1 release!

That's sort of fast.

[Anonymous Coward]

I guess.

Re:That's sort of fast.

[jackbird]

640 GB should be enough for anyone.

Re:

[networkBoy]

Congrats.
For the first time in ages that joke was funny. ;-)

Also building a slightly slower computer...

[AceCaseOR]

IBM is also building a slightly slower computer, called "Wile E. Coyote", which is slightly slower. They are currently attempting to work out the bugs, as it keeps crashing...

Re:Also building a slightly slower computer...

[steveo777]

They're currently thumbing through ACME cataloges for spare rockets and rollerskate. That should speed things up.

However part of the problem

[OzPeter]

has been identifed as sub-standard components delivered by a third party company called "acme".

These components had a tendency to either explode at in-opportune moments, or behave in a manner that while was true to the letter of their description was totally ineffective for the desired purpose.

At the moment each side is gathering its hoards of lawyers and all involved are jumping up and down, waving thigh-bones in the air and screaming incomprehensible abuse at each other.

Re:

[mickwd]

has been identifed as sub-standard components delivered by a third party company called "acme".

These components had a tendency to either explode at in-opportune moments, or behave in a manner.....

You said "acme" - I think you meant "Sony".

Re:

[Don853]

It's been too long since you've watched Saturday morning cartoons, spoilsport.

Re:However part of the problem

[reverseengineer]

The unique failure modes of these Acme components, however, have caused IBM researchers to stumble upon a remarkable macroscale quantum effect which may be useful in the development ofquantum computers.

Apparently, when the Acme Rocket Sled, Acme Giant Rubber Band or the Acme Bat-Man suit reach their point of failure, every particle of the unfortunate user is compelled into a quantum superposition (known as the Chuck Jones state) where the particles of the user appear to exist outside of the normal flow of time, during which the user can apparently communicate with the outside using messages written on signs. The wavefunction collapses, however when the user realizes the peril of the current situation; the user returns to normal time and is contacted catastrophically by the approaching train/TNT detonation/boulder/ground/ground followed by a pursuant boulder.

IBM scientists believe that useful calculations could be made nearly instantaneously from the perspective of outside observers, if only the user inside the Jones state could be induced to work complex math problems and write the answer on a picket sign, rather than simply using such signs for messages like, "Why Me?", "Not Again!", "?!?!?!?!?!" or "Ouch."

NASA is also working with Acme to determine the physical mechanism by which the Acme Portable Hole functions.

Re:

[Irish_Samurai]

Insightful? Now that is even funnier than the post.

Re:

[adam31]

These components had a tendency to either explode at in-opportune moments

Good Lord! Another /. article bogs down into Sony bashing...

I see through your 'acme' shenanigans.

PS3 delayed?

[October_30th]

So, is this the reason why the PS3 release has been delayed?

Re:

[networkBoy]

Sony did say they were having trouble acquiring key components on the assembly line...
Maybe not so much a joke after all?
-nB

Re:PS3 delayed?

[evanbd]

No. Sony said it was due to shortages of the blue laser diode in the Blu-ray drives. Also, you'll note they're short a couple million PS3s, not a measley 16000.

Re:

[evanbd]

Eh, while in general I agree with you, there are two other factors here. First, they really are short millions, not tens of thousands, of units. I have trouble coming up with simple reasons why they'd lie about that (apply Occam's razor for the rest of the argument). And second, they do have a certain amount of fiduciary responsibility to their shareholders (of non-zero but decidedly not infinite impact), which leads to a desire to place blame when possible (well in line with human nature as demonstrated

Re:

[__aaclcg7560]

First, there's a shortage of blue diode lasers. Now, there's a shortage of Cell processors. If you were expecting a Sony discount before Christmas, forget about it.

Flops? CPS?

[Cybert4]

For one, they gloss over whether they mean floating point operations or "calculations" per second. The article seems to equate a flop with "calculations per second". The flop, of course, came from floating point operation. Even then it's vague--is it single, double or double-extended?

Yes, it's certainly better than the old "megahurts" races. But I think they could come up with something better.

Re:

[Anonymous Coward]

> Yes, it's certainly better than the old "megahurts" races. But I think they could come up with something better.

Bogomips? [wikipedia.org]

Definitely flops

[Junta]

Though not necessarily 64-bit precision flops, as are required for top500 scores... The cell isn't impressive double-precision wise.

PR Numbers

[scoobrs]

Here's an explanation. Keep this in mind whenever you read PR about vapor hardware... Most likely the confusion between FLOPS and "calculations per second" is not unlike the confusion between peak PR numbers, peak Linpack results, sustained Linpack results, and sustained application FLOPS. For example, no Cell processor ever reaches the impossible speed of 360 GFLOPS on any real world scientific application because of the real world problems of a slow interface to memory, storage, network, etc. which all

Re:Flops? CPS?

[adam31]

The flop, of course, came from floating point operation. Even then it's vague--is it single, double or double-extended?

I was thinking the same thing. Running the numbers, 256 GFlop * 16,000 => 4.096 PFlop @ single precision. So if IBM means SP flops, something is slowing its theoretical max down by 2.5x. But Cell's DP perf yields 18.2 * 16,000 => .292 PFlop @ DP. So that's not it either.

It's long been rumored that a post-PS3 Cell is in development that can pipeline DP flops. Its max theoretical DP perf would still be half of SP because it's just 2 DP values per 128-bit register instead of 4. AND, if you figure they lower the GHz to 3.2 to cut the heat output in half, you arrive at the magical number... 1.638 PFlop.

So can we take this as evidence that there now exists a Cell that performs DP calculations pipelined?

Feet/Metres/Meters

[onion2k]

Roadrunner is to be installed at Los Alamos National Laboratory, occupying 1,100 square metres of floorspace (that's a square about 110 feet on a side)
Why mix the units like that? It's either 33 meters a side, or its 12,100 square feet. Mixing units is the sort of thing that can only lead to errors.

And for the record, sqrt(1100m2) = 33.17 meters = 108.83 feet a side. 110 feet per side gets you an extra 24.13 square meters .. enough for 4 interns including desks.

Re:Feet/Metres/Meters

[lightyear4]

And for the record, sqrt(1100m2) = 33.17 meters = 108.83 feet a side. 110 feet per side gets you an extra 24.13 square meters .. enough for 4 interns including desks.

You mean in the room WITH the supercomputer? Oooh! I call dibs on the sauna office!

Re:

[DocSavage64109]

Those 1.17 x 54 foot spaces sound like the ideal workspaces for interns.

You missed the word "about"

[everphilski]

occupying 1,100 square metres of floorspace (that's a square about 110 feet on a side) It is not abnormal to present both english and metric units in a presentation such as this when you have a diverse audience (the source was the BBC, although it appears the article submitter did his own math to get the side length). 110-108.83 = 1.17 meters, which is 1.06% off. Which is more than acceptable when talking to the "common man" ... now when ordering the carpenting, on the other hand ...

Re:

[jelle]

"Why mix the units like that? It's either 33 meters a side, or its 12,100 square feet. Mixing units is the sort of thing that can only lead to errors."

'Non-metric people' are used to units being mixed up... The solution taken by many is to either give up and think that 'math is difficult', or to only use rounding/approximations for 'quick calculations': '5000 feet per mile' (instead of 5280)...

Rounding like that is what results in allowing space for the 4 interns... Hurrah for rounding: Where would the inte

Re:

[OzPeter]

By the way .. there are also two meanings of gallon .. US and Imperial

Re:Feet/Metres/Meters

[raehl]

Why mix the units like that?

Training to be a rocket scientist?

Re:

[gstoddart]

"Roadrunner is to be installed at Los Alamos National Laboratory, occupying 1,100 square metres of floorspace (that's a square about 110 feet on a side)"
Why mix the units like that? It's either 33 meters a side, or its 12,100 square feet. Mixing units is the sort of thing that can only lead to errors.

Well, the original article says it's about 12,000 square feet, but it's the BBC, so they nicely give an approximate in metric of 1,100 square metres since more of their readers might know what that means.

The po

Re:

[tygerstripes]

In my defence, I didn't include the botched translation into feet in the original submission. After all, I'm from the UK - it would be yards or Olympic swimming pools.

Re:

[be-fan]

You didn't specify gender...

So the price was

[CrazyJim1]

16,000 *600$= 9.6 million. That doesn't seem like much for the biggest super computer.

Re:

[patio11]

Hah, hah, hah.

Nope, things don't scale linearly, most especially not when you're talking government contracting. For comparison, Red Storm at Sandia National Labs cost $90 million. http://www.techcommjournal.org/PDFSVol3No3/16topte nTC11.pdf [techcommjournal.org] Japan's Earth Simulator was about a quarter billion.

Re:

[CrazyJim1]

Whats even funnier is that I got modded insightful when I was just counting the cost in Playstations. Heh, Slashdot is funny sometimes.

Re:

[patio11]

Maybe folks were giving you +1 for "If you have a national nuclear R&D budget to spend, you too can afford a PS3?"

Other government bodies that could buy a PS3:

NASA, but they'd crash it into something and want a new one.

The IRS, but it would depreciate to 20% of its value the day they bought it, unless they sold it on eBay in case its fair value would be $10,000 regardless of the auction final price.

The Marines, except they play Wii, because Marines will only touch a console made for Real Men (TM).

The Na

Re:

[moogleii]

You've covered the price of one set of 16,000 chips, kinda. What about the other 16,000? And that's just the chips. There's power, cooling, the rest of the hardware, software...

Re:So the price was

[gstoddart]

16,000 *600$= 9.6 million. That doesn't seem like much for the biggest super computer.

First off, I believe IBM manufactures the cell processors for Sony, so it probably didn't cost them that much for the actual processors.

But, don't forget the $58 million in IBM consultants who built the damned thing. That's the real cost of this. ;-)

Cheers

Re:

[1lus10n]

Believe it or not, we are not ALL consultants at big blue. Some of us actually test, configure and ship it out for consultants to *polish* it.

And yes by *WE* I am including myself and the people I work with on this project. (and yes incase you didnt figure it out, I work at IBM)

Re:

[Pharmboy]

Believe it or not, we are not ALL consultants at big blue.

So are you a lawyer or in marketing? ;)

State Bird

[jbeaupre]

The roadrunner is also the state bird of New Mexico, location of LANL.

http://en.wikipedia.org/wiki/Roadrunner_(bird) [wikipedia.org]

It was always ironic to see them running up and down the road in front of my grandparents home.

Re:

[pilgrim23]

there were a lot more to see back when LANL was LASL :)

Re:

[Splab]

Roadrunner is also the nickname for the guy/gal who manages to get the fastest speedup in the cluster computing and architecture course taught at DIKU (www.diku.dk). (And to top off the fun, each assignment is based around the roadrunner cartoons)

Re:

[geekoid]

Ironic is not coincident.

Re:

[Just Some Guy]

It was always ironic to see them running up and down the road in front of my grandparents home.

Was it because your grandparents lived in Vermont? Come on - don't leave us in suspense!

Would you like to play a game?

[mendaliv]

Interesting sidenote in the article not mentioned here:
"The laboratory is owned by the US Department of Energy (DOE). Eventually the machine could be used for a programme that ensures the US nuclear weapons stockpile remains safe and reliable, the DOE said in a statement."

Why do I get a weird feeling that I've seen this sort of thing in one too many movies?

Re:

[Rob T Firefly]

*draws service revolver*

Turn your key, sir!

Re:

[Kesch]

Yep, one of the big uses of Roadrunner is to simulate nuclear reactions. We have quite a few nukes that have passed their expiration date and we don't fully understand how they will function. Instead of just blowing a few up to find out, we'd rather use a simulation.

Re:

[1lus10n]

not really, but stupid international oversight and treaties got in the way.

Re:

[Glonoinha]

Probably a shitload cheaper to set off a nuke than to simulate it, too, come to give it some serious consideration.
Just crack the seal on one, turn it loose and a'splode it - see what happens.
Maybe schedule it for the 4th of July next year, get a double bang for your buck.

What they are doing now is akin to buying a gazillion dollars worth of Ben & Jerry's ice cream, keeping it in the freezer for four decades and then spending another gazillion dollars in software and hardware to determine if it still ta

Um, no, not exactly

[$RANDOMLUSER]

From TFC (caption):

The cell processor was originally designed for Sony's PlayStation 3

I'm sure this comes as a surprise to IBM and Toshiba.

And this is from BBC News, no less. <sigh>

Why should IBM be surprised?

[Khyber]

They were responsible for the development of the Cell processor alongside Sony.

Re:

[$RANDOMLUSER]

Don't know anything about Cell, do you?
Here [ibm.com]'s the original press release.
And here [ibm.com]'s the ISSCC "opening the kimono" press release.
Funny, lots of talk about "supercomputer on a chip", no mention of PS3.
This application is EXACTLY what IBM has been talking about for this chip all along. And they did all the work.

Re:

[oscartheduck]

From the original press release: " Sony Computer Entertainment Inc. (SCEI), IBM Corporation (IBM) and Toshiba Corporation (Toshiba) announced today plans to research and develop an advanced chip architecture for a new wave of devices in the emerging broadband era."

While it doesn't mention the Playstation 3, the mention of Sony being involved would certainly seem to indicate that Playstation 3 was considered.

Re:

[nutshell42]

Just take a look at its design. It has EmotionEngine on steroids written all over it.

Re:

[Doctor Memory]

IBM brings the Fab Brians

So who are these guys named Brian, and what's so special about them?

Re:

[raftpeople]

IBM brings the Fab Brians + other skills

Actually, IBM brings the CPU design brains more than the fab. They offer the service of designing custom processors and they have much experience in this arena, plus they happen to have PowerPC available for the controlling proc.

Lame

[Wolvie MkM]

And still it only runs F.E.A.R. at 25fps... weak...

what about Blue Gene/P?

[tubbtubb]

I thought BlueGene/P was targeting a petaflop?
I don't think this Cell based thing is its replacement. If BGP is still coming, it should be coming soon:
link [spscicomp.org]

And looking back 20 years from now

[syntap]

we'll laugh at such a large room full of computer equipment, the equivalent of which will be powering our mobile communications devices in a 150mm x 150mm package.

Re:And looking back 20 years from now

[cowscows]

Yeah, I've decided that I'm not buying an iPod until apple wises up and adds the ability to recieve FM radio, and simulate the aging of my large nuclear weapon stockpiles.

20 years from now behind a horse...

[mrraven]

You hope... Or we could be hitching horses to a buckboard to ride into town when the cheap oil runs out and there is no good energy substitute available.

Re:

[mrraven]

Such as? Before you answer consider the energy cost building the entire infrastructure system.

Re:

[iggymanz]

or we'll be running Linux but the honkin' bloated Sun jvm brings our pocket supercomputers to its knees

Sure it's fast

[Skraut]

But is it fast enough to figure out the Answer to Life, the Universe, and Everything?

Re:

[PixelSlut]

Oh yeah, that's the easy part. Next they're going to build an even more powerful computer that's going to compute the question!

Link to IBM's Cell SDK

[DigitalDreg]

http://www-128.ibm.com/developerworks/power/cell/ [ibm.com]

The toolchain and a simulator are freely available and run on Fedora Core 5 systems. Take a look for yourself.

I wonder....

[mhore]

Is this IBM's new approach to clusters/supercomputers? We just bought a relatively small (i.e. 176 processors) system from them... half is Power5+ and the other half is Xeon.

Hmmm

[Klaidas]

It's big. It's powerful. But, in simple human talk, how much GHz does that thing have?

Re:

[suggsjc]

Forgive the oversimplification...
GHz is a measurement of how fast the clock cycle of a processor is. This system will have lots of processors, that will contibute to the computing power (# of flops [floating point operations]) of the overall system. So, GHz isn't a good measurement. However, I'll try to give you a meaningless comparison.

From wikipedia:

A relatively cheap but modern desktop computer using, for example, a Pentium 4 or Athlon 64 CPU, typically runs at a clock frequency in excess of 2 GHz

Big deal

[I Like Pudding]

I can do 1.8pflops with a #2 pencil, some scratch paper, and a few grams of peyote.

My favorite line in TFA

[jthill]

IBM says it will start shipping the new supercomputer later this year.

It took me a beat to get it.

I'm surprised this wasn't tagged "AMD" sooner...

[The Last Gunslinger]

According to today's Austin American Statesman article [statesman.com] , the other 16,000+ CPUs in this machine will be AMD Opterons.


And, the article also confirms that the machine will indeed be running Linux.

Re:Billion or billion?

[morgan_greywolf]

British 'billions' are American 'trillions'. So this may be running three orders of magnitude faster than you first expec

Yeah, go figure. Stupid Brits can't even speak English. ;)

Re:

[Firehed]

Huh? A million is a thousand times a thousand. A billion is a thousand times a million. A trillion is a thousand times a billion. Continue with quadrillion, quintillion, etc. Note the prefixes here (mi is mono, bi, tri) indicate how many thousands of times larger they are than a thousand, or I suppose sets of three orders of magnitude larger if you insist (one three-digit seperator's worth). I think this is the one case where our logic is more consistant. I'm sure there's a more technical term for su

Re:

[Firehed]

looks like the decimal number system is more fucked than i thought, thanks for destroying the illusion!

Glad to be of service. Thank you, come again!

Re:

[Tremor (APi)]

I think they've switched over, from Wikipedia [wikipedia.org]: "Short scale is the English translation of the French term échelle courte, which designates a system of numeric names in which the word billion means a thousand millions.

Long scale is the English translation of the French term échelle longue, which designates a system of numeric names in which the word billion means a million millions.

For most of the nineteenth and twentieth centuries, the United Kingdom uniformly used the long scale, while the United

Re:

[Gothmolly]

Meanwhile, notice that its the smelly, cheese-eating surrender monkeys who farked the whole thing up in the first place.

Shouldn't be a problem unless...

[The_REAL_DZA]

...we're planning on sending it to Mars [wikipedia.org].

Re:

[Curmudgeonlyoldbloke]

No - I've been around a few years and was never taught "billion = million million" at school (and I do remember changing from pounds, shillings and pence, so that should help you put a date on it).

I think that the "official" changeover (as far as the treasury was concerned) was late 60s / early 70s. A quick google can't find a cite for it but a post here mentions "the official announcement some three decades ago":
http://66.249.93.104/search?q=cache:O4P5O5xh-6sJ:w ww.translatorscafe.com/cafe/MegaBBS/thread- [66.249.93.104]

Re:

[maxwell demon]

That's the opposite of an improvement: With the "old British" system you need to learn less words for the same number range. You just have to remember that a thousand "-ions" are one "-iard".

Moreover it's simpler: From the billion on, the prefix is named after latin numer names: bi->2, tri->3 etc.
With the "old British" system, an "n-ion" is a million to the power of n. Quite simple.
With the US system, an "n-ion" is thousand to the power of (n+1). An extra complication. Not much, but still.

Re:

[stevesliva]

That's the opposite of an improvement: With the "old British" system you need to learn less words for the same number range. You just have to remember that a thousand "-ions" are one "-iard".

I prefer a system of prefixes, only. If your statement of -iard as a suffix is true, the wikipedia entry [wikipedia.org] needs to be updated with trilliards, quadrilliards, quintilliards, etc.

Re:

[maxwell demon]

If your statement of -iard as a suffix is true, the wikipedia entry needs to be updated with trilliards, quadrilliards, quintilliards, etc.

Indeed.

Re:

[maxwell demon]

It is too an improvement. I'm British and I've never even heard the word "milliard" so we can see how successful that system was in education.

That's a complete non-argument: The reason the system changed in GB is probably being flooded with "US-billions", and the absolute worst thing is if you never know if the "billion" you read is 10^9 or 10^12. And the fact that you never heared "milliard" may simply be due to not being old enough.

Also note that in France the "-ion/-iard" scheme was the successful one, d

Re:

[ScrewMaster]

No, but it can crack fifteen billion iTunes files per second.

Re:

[ZakuSage]

Unless IBM happens to be using blue lasers in their new supercomputer, it has absolutely nothing to do with PS3's launch numbers.

Re:

[KarmaMB84]

They don't call them Big Blue for nothing.

Re:

[maxwell demon]

If I could get a petaflop supercomputer the size of a George Foreman Grill for $600, I could live with the missing HDMI cable. :-)

Re:

[Openstandards.net]

Excellent and informative post. Thank you. Can you explain what SPE is, please? Also, can you describe the installation and integration of Linux? I'm curious how someone can brew a home version, on a much smaller scale, of course, perhaps 16 CPUs, but using a similar architecture.

Re:

[Kesch]

I'm working on the project right now, and I can assure you that we are only building this so we will be ready for Crysis.

Re:

[somersault]

"I hope you'll be happy when you run Linux and its cli interface"

Yeah, why do they use all those buttons in planes and stuff!!!?!?!! It would surely be a lot more efficient if they just used a one button mouse to fly them? In fact, get rid of the buttons altogether!! Evil!

Re:

[oscartheduck]

YOu have a source on that flamebait?

Re:

[1lus10n]

It doesnt have to use power5, its got cell.

Re:

[1lus10n]

http://www.mc.com/cell/products/view/index.cfm?id= 96&type=Boards [mc.com]

Thats about the closest your going to get at this point.

Re:

[1lus10n]

I believe you do not know what you are talking about. The reason this cluster has so many opterons is because they are double precision and has such are better suited for doing the task at hand, the cell's are used for specialized tasks only in this implementation. The Cell architecture is not going to be limited to just one type of application, it is designed with everything from supercomputing to embeded stuff in mind, its just going to take a while to get to the point where there are chips of varrying ty

Re:

[raftpeople]

6809 programming in machine code, me too. I couldn't afford an assembler.

I actually enjoy the process of figuring out how to program for something like the cell, but once I've been through it once, then I don't want to have to do it that way anymore. The joy is in solving the problem.