Slashdot Log In
With Linux Clusters, Seeing Is Believing
Posted by
Hemos
on Mon Dec 13, 2004 11:41 AM
from the believing-is-truth dept.
from the believing-is-truth dept.
Roland Piquepaille writes "As the recent release of the last Top500 list reminded us last month, the most powerful computers now are reaching speeds of dozens of teraflops. When these machines run a nuclear simulation or a global climate model for days or weeks, they produce datasets of tens of terabytes. How to visualize, analyze and understand such massive amounts of data? The answer is now obvious: using Linux clusters. In this very long article, "From Seeing to Understanding," Science & Technology Review looks at the technologies used at Lawrence Livermore National Laboratory (LLNL), which will host the IBM's BlueGene/L next year. Visualization will be handled by a 128- or 256-node Linux cluster. Each node contains two processors sharing one graphic card. Meanwhile, the EVEREST built by Oak Ridge National Laboratory (ORNL), has a 35 million pixels screen piloted by a 14-node dual Opteron cluster sending images to 27 projectors. Now that Linux superclusters have almost swallowed the high-end scientific computing market, they're building momentum in the high-end visualization one. The article linked above is 9-page long when printed and contains tons of information. This overview is more focusing on the hardware deployed at these two labs."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Realisation about this procedure (Score:5, Funny)
Re:Realisation about this procedure (Score:2, Funny)
Re:Realisation about this procedure (Score:2, Funny)
Re:Realisation about this procedure (Score:3, Funny)
uh-oh, more bad pick-up lines for Linux Geeks:
"you don't need to imagine how big my beowulf cluster is"
Re:Realisation about this procedure (Score:4, Funny)
And it's always the guys with small clusters who say that size doesn't matter?
Parent
Mac OS X has similar benefits (Score:4, Interesting)
By contrast, NCSA's surprise entry in November 2003's list, Tungsten, achieved 9.82 Tflops for $12M asset cost.
Double the cost, for a Top 100 supercomputer's-worth lower performance.
And it wasn't because Virginia Tech had "free student labor": it doesn't take $6M in labor to assemble a cluster. Even if we give it an extremely, horrendously liberal $1M for systems integration and installation, System X is still ridiculously cheaper.
I know there will be a dozen predictable responses to this, deriding System X, Virginia Tech, Apple, Mac OS X, linpack, Top 500, and coming up with one excuse after another. But won't anyone consider the possibility that these Mac OS X clusters are worth something?
Re:Mac OS X has similar benefits (Score:4, Informative)
Parent
Re:Mac OS X has similar benefits (Score:5, Insightful)
We at Terra Soft have just released Y-HPC, our version of Yellow Dog Linux, with a full 64-bit development environment, and a bunch of cluster tools built in.
I'm not much of a marketting drone, but being as I am part of the Y-HPC team, I had to put a shameless plug in. Bottom line is, it kicks OSX's ass any 2 ways you look at it.
Y-HPC [terrasoftsolutions.com]
Parent
Re:Mac OS X has similar benefits (Score:3, Insightful)
I've tried installing YDL on a small G5 cluster. It was a PITA to get running (3 installs before I was able to get the X server running right). And still I can't find any fan control. After 5 minutes the fans spool up to "ludicrous speed" and stick there.
I really want to like YDL. I've been talking to the folks who do OSCAR about trying to get OSCAR to support YDL. But I'm not sure how it will work out yet, at least until I can figure out how to turn down the fans!
Re:Mac OS X has similar benefits (Score:4, Informative)
Fan control was integrated into the kernel over a month ago, and is most definitelly in the first version we released last week.
We have also developed a nice pretty installer for the head node in a cluster, and wrote Y-Imager (front end for Argonne's System Imager), to automate the building of compute nodes in a cluster.
No offense taken
Parent
Re:Mac OS X has similar benefits (Score:4, Insightful)
Parent
Re:Mac OS X has similar benefits (Score:5, Informative)
By contrast, NCSA's surprise entry in November 2003's list, Tungsten, achieved 9.82 Tflops for $12M asset cost.
When I looked here [uiuc.edu], I found this: ``Tungsten entered production mode in Novermber 2003 and has a peak performance of 15.36 teraflops (15.36 trillion calculations per second).''
To me, that looks faster than System X, not slower.
Let's see: NCSA stands for ``National Center for Supercomputing Applications''. ``NCSA [uiuc.edu] is a key partner in the National Science Foundation's TeraGrid project, a $100-million effort to offer researchers remote access ...''
Looks as if the NCSA has a huge budget. I'd guess that ``gold-plated everything'' and ``leave no dollars unspent'' are basic specs for everythig they buy.
What can we learn about Virginia Tech? How about this [vt.edu]:
In addition to the volunteer labor, I'd guess that Virginia Tech had very different design goals, in which price was a factor. NCSA's bureaucracy probably accounted for a lot of those extra $6M they spent. Different designs and goals probably had a lot to do with the rest of the price, but I suspect that a bureaucratic procurement process was the main cause for the higher price of the Xeon system.Yes, System X and the Apple hardware is pretty neat, but don't use the price/performance ratio of these two systems as a metric for the relative worth of Linux and OSX clusters.
It's unfair and meaningless to compare volunteer labor and academic pricing and scrounging on a limited budget to bureaucratic design, bureaucratic procurement and an unlimited budget.
Parent
Rpeak, not Rmax (Score:5, Insightful)
The speed you quoted is the theoretical peak, not the actual maximum achieved in a real world calculation (like the Top 500 organization's use of Linpack).
System X's equivalent theoretical peak is 20.24 TFlops.
I'm also not indicting Linux clusters in the least; they've clearly shown they can outperform traditionally architected and constructed supercomputers for many tasks, with the benefit of using commodity parts - at commodity pricing. All I'm saying is that there's a new player here, and it's a real contender, and has done a lot for very little money...which was the whole goal of Linux clusters in this realm in the first place.
(Also, as I said, the volunteer labor model is irrelevant - let's just pretend it was professionally installed for an additional $1M, or even $2M if that would satisfy you. It's still several million dollars cheaper, and 3Tflops greater performance. These are BOTH rackmount clusters with similar amounts of nodes and processors, running a commodity OS with fast interconnects. There are differences, yes, and perhaps even differences in goals. But looking past that, price/performance for something like this is still an important metric.)
Parent
Re:Mac OS X has similar benefits (Score:2, Insightful)
Vendors will often give away hardware in order to
Re:Mac OS X has similar benefits (Score:5, Insightful)
Your right!
1st, System X or the "Big Mac" was thrown together so that people like us would talk about it and to get a good standing for the November 2003 top 500 list. They did an excellent job at this.
Now for some reality. The system is not yet operational.
When it was first thown together, everyone "in the know" and myself questioned how this was going to work without a reliable memory subsystem, and the VT people responded that they were going to write software to correct any hardware errors, and we said OK, whatever. Then, they said, hmm, we kinda needa a reliable memory subsystem, so lets rip out all 1,100+ machines and start over with these new Xserve boxes that have ECC memory in them.
This system has not come up yet with the new Xserves, according to their website [vt.edu].
Now, I'm going to make a comment on Linpack. Linpack, like all good benchmarks are really good at measuring that benchmark's performance. Linpack is a good benchmark, but it is also a benchmark that does not require much RAM per node to run. Some applications do need a good amount of RAM/node to run and being that RAM costs $$, the cost adds up very quickly, and the cost/cpu/teraflop goes down accordingly.
With the comparison between System X and Tungsten NCSA cluster. Personally, I don't know why the Tungsten cluster cost more because the Mac cluster has more RAM/node and each node should have been cheaper in general. The NCSA cluster uses Myrinet which I know is expensive, but I do not know that in comparison to the Infiniband equipment on the Macs. Supposedly, the Infiniband interconnects were what got System X on the top500 list with such good results, or at least that is what the head of the project told me.
Although its popular here on slashdot because many of the readers are younger and inexperienced (and have no money) that they praise anything that costs less and extra brownie points go towards an underdog like AMD or Linux, however in the real world people actually will pay extra for something to ensure that it works. Working equipment may seem superfluous to the dorm room Linux guru, but trust me, I know what its like to work with equipment that cost about $1 mil and it doesn't work. We could have gone with the 2nd bidder at $1.2 mil and it would have worked. Yes, we "saved" $200,000, but we also wasted well over $500,000 when one considers that over 50% of the equipment is faulty and many people's time has been wasted.
Parent
Re:Mac OS X has similar benefits (Score:2)
Yeah, VT really didn't do anything... (Score:4, Interesting)
Parent
You dropped something (Score:2, Funny)
How to write complete sentences?
not suitable for the Slashdot demographics (Score:3, Funny)
I hope the poster doesn't actually expect any of us to post any meaningful comments based on having read that article, it's a lost cause.. At least on me.
Is that US or metric tons? (Score:5, Funny)
Damn! What kind of paper stock are you printing on?
Re:Is that US or metric tons? (Score:3, Funny)
Re:Is that US or metric tons? (Score:3, Funny)
Roland Piquepaille and Slashdot (Score:5, Interesting)
I think most of you are aware of the controversy surrounding regular Slashdot article submitter Roland Piquepaille. For those of you who don't know, please allow me to bring forth all the facts. Roland Piquepaille has an online journal (I refuse to use the word "blog") located at www.primidi.com [primidi.com]. It is titled "Roland Piquepaille's Technology Trends". It consists almost entirely of content, both text and pictures, taken from reputable news websites and online technical journals. He does give credit to the other websites, but it wasn't always so. Only after many complaints were raised by the Slashdot readership did he start giving credit where credit was due. However, this is not what the controversy is about.
Roland Piquepaille's Technology Trends serves online advertisements through a service called Blogads, located at www.blogads.com. Blogads is not your traditional online advertiser; rather than base payments on click-throughs, Blogads pays a flat fee based on the level of traffic your online journal generates. This way Blogads can guarantee that an advertisement on a particular online journal will reach a particular number of users. So advertisements on high traffic online journals are appropriately more expensive to buy, but the advertisement is guaranteed to be seen by a large amount of people. This, in turn, encourages people like Roland Piquepaille to try their best to increase traffic to their journals in order to increase the going rates for advertisements on their web pages. But advertisers do have some flexibility. Blogads serves two classes of advertisements. The premium ad space that is seen at the top of the web page by all viewers is reserved for "Special Advertisers"; it holds only one advertisement. The secondary ad space is located near the bottom half of the page, so that the user must scroll down the window to see it. This space can contain up to four advertisements and is reserved for regular advertisers, or just "Advertisers". Visit Roland Piquepaille's Technology Trends (www.primidi.com [primidi.com]) to see it for yourself.
Before we talk about money, let's talk about the service that Roland Piquepaille provides in his journal. He goes out and looks for interesting articles about new and emerging technologies. He provides a very brief overview of the articles, then copies a few choice paragraphs and the occasional picture from each article and puts them up on his web page. Finally, he adds a minimal amount of original content between the copied-and-pasted text in an effort to make the journal entry coherent and appear to add value to the original articles. Nothing more, nothing less.
Now let's talk about money. Visit http://www.blogads.com/order_html?adstrip_category =tech&politics= [blogads.com] to check the following facts for yourself. As of today, December XX 2004, the going rate for the premium advertisement space on Roland Piquepaille's Technology Trends is $375 for one month. One of the four standard advertisements costs $150 for one month. So, the maximum advertising space brings in $375 x 1 + $150 x 4 = $975 for one month. Obviously not all $975 will go directly to Roland Piquepaille, as Blogads gets a portion of that as a service fee, but he will receive the majority of it. According to the FAQ [blogads.com], Blogads takes 20%. So Roland Piquepaille gets 80% of $975, a maximum of $780 each month. www.primidi.com is hosted by clara.net (look it up at http://www.networksolutions.com/en_US/whois/index. jhtml [networksolutions.com]). Browsing clara.net's hosting solutions, the most expensive hosting service is their Clarahost Advanced (http://ww [clara.net]
Re:Roland Piquepaille and Slashdot (Score:2, Offtopic)
Oh, please, you give Roland WAY too much credit. He doesn't add any original content. He just copies and pastes.
Re:Roland Piquepaille and Slashdot (Score:3, Informative)
Are you actually Roland Piquepaille? If so, that's a really neat trick to move traffic to that site. If not, then he may be thankful for your comment, after all :-)
30 accepted stories since August 29th, 2004! (Score:5, Interesting)
Parent
Re:Roland Piquepaille and Slashdot (Score:5, Funny)
Parent
Wow! (Score:3, Funny)
Obligatory (Score:3, Funny)
Re:Wow! (Score:2)
When Harlie was one [amazon.com] was the first book that I recall about a computer that designed another more complex computer that only it could understand.
Maybe Harlie was a Linux cluster.
Re:Wow! (Score:2)
Ok, now we just need another supercomputer to test the supercomputer the supercomputer built us to interpret the output of the supe
Big Screen! (Score:2)
Re:Big Screen! (Score:3, Informative)
I know you're joking, but since I'm the hardware architect for the LLNL viz effort, I'll bite anyway. :-)
Here's what you'll need at minimum:
Regarding the story title (Score:3, Funny)
Does this mean that we don't have to just imagine a Beowulf cluster anymore?
You are correct, sir (Score:4, Funny)
Parent
Finally.... (Score:3, Funny)
You would think so (Score:3, Informative)
Fuck Roland Piquepaille (Score:5, Insightful)
Building clusters with linux is easy. (Score:4, Interesting)
Re:Building clusters with linux is easy. (Score:4, Interesting)
Computers were initially monolithic machines that effectively had a single core. By the 70's, the processing on many mainframes had branched out so that a single mainframe was often a number of seperate systems integrated into a whole (though nothing on the level we see today). By the 80's it seemed to swing back to monolithic designs (standalone pc's, ubercomputer Crays) and it wasn't until the 90's that dual and quad processing became commonplace (though the technology had existed before).
Eventually, someone will hit on a revolutionary new technology (sort of like how transistors, IC's, and microprocessors were revoloutionary) that renders current LVSI systems obsolete (optical? quantum?), and the cost/power ratio will shift dramatically, making it more economical to go back to singular (and more expensive) powerful cores rather than cheap (but weaker) distributed cores.
Parent
Re:Building clusters with linux is easy. (Score:2, Insightful)
very long article... (Score:4, Interesting)
Really... (Score:5, Insightful)
While some simulations parallelize very well to cluster environments, there are still plenty tasks that don't split up like that.
The reason clusters make up a lot of the Top 500 list is that they are relatively cheap and you can make them faster by adding more nodes - whereas traditional supercomputers need to be deisgned from the ground up.
mail address (Score:2)
Not fair, Linux! (Score:2, Funny)
Single image shared vs distributed memory in Linux (Score:4, Insightful)
From imagining to understanding... (Score:3, Insightful)
Graphic Card Technology (Score:3, Interesting)
Namely, it allows for graphics cards to operate better in situations exactly like this; clustered applications. As it stands, the graphics card can crunch an enormous amount of data, but is extremely poor at sending it back to the CPU & system. It's optimized for screen dumping only.
Sony's Cell is going to be absolutely crucial as a tech demo for this foresighted technology. We're heading towards a more distributed computer architecture where various specialized units pipe data between each other.
In summation,
Its my hope that eventually graphics cards will catch up and perform better bi-directionally. After that, we've got to wait another 5 years for PCI-E implementations to catch up and perform better switching (vis-a-vise multiple fully-switched x16 busses). We are moving away from the CPU for high performance computing; the cpu currently performs both control and data-processing. Graphics cards are just the first wave of the distributed architecture phenomena, Cell will be a light-year jump towards the future of computing in the intricate levels of hardware reconfigurability. there's a good powerpoint on the patents behind cell here [unc.edu].
Ultimately this will lead towards the tearing down of the computer as a monolithic device, and a rethinking of what exactly the network and os's roles are. Queue exo-kernel and DragonFly BSD debates.
Scientific community is so stupid (Score:3, Funny)
Re:Uh huh ... (Score:3, Insightful)