Building Your First Cluster?

An anonymous reader asks: "I'm interested in building a DIY cluster using Linux and will be using conventional Linux software. However, the number of possible ways to do this is huge. Aside from Beowulf, there's Mosix, OpenMosix, Kerrighed, Score, OpenSSI and countless others. Therein lies the problem. There are so many ways of clustering, development seems to be in fits and starts, most won't work on recent Linux kernels and there's no obvious way to mix-and-match. What have other people used? How good are the solutions out there?"

Whatever happened to...

[techno-vampire]

I remember in the '80s, back at JPL, they were talking about connecting a number of computers into a hypercube. One node would be for I/O, and it would communicate with the others along the edges of a hypercube.

Two computers make a 1 dimensional "cube." Four, in a round-robin make a square. Six, properly connected, make a regular cube and so-on. Does anybody out there know if they still connect clusters this way and if not, why?

I am in the exact same position.

[B5_geek]

I have 5 boxes of somewhat random configuration that I am looking to do the same thing as the OP. My goal for this cluster would be high-speed video processing (using tovid).

sweating it out..

[tempest69]

ok a few caveats (I cant spell)

1. I do beowulf, other clusters arent my thing. 2. I can handle C and C++, but I'm not a guru. 3. I can fumble my way through unix-linux but I get cranky with new versions (command/ flag changes in utilities). 4. I have 6 lazyish years as a unix sysadmin.

getting prepped,

You want to make sure that the boxes that are talking to eachother are very secure from the rest of the world. Most of the concepts on a cluster are about trashing the security of the machines in question. There are ways to make a secureish cluster, but a good firewall is a better way to go. let the firewall talk to your "head node", and preferably locks the "body nodes" from seeing the outside world. There are a ton of ways to get this done. on the cheap have all the body nodes have a non-existant gateway ie 192.168.0.1, set the firewall as 192.168.0.129 (forget dhcp) and let the head node point to the 192.168.0.129,, and have the firewall route services (ssh, ftp, telnet (ok not ftp or telnet)) to the head node.

getting started

1. Load all the boxes with the same OS. (the same way) (DONT select SELINUX or you will cry) 2. build a hosts file (names for machines) /etc/hosts 3. build a hosts.allow , hosts.equiv (still in /etc) 4. add in some entries into securetty for your rlogin rcp rsh.. 5. youll probably have kerberos(weakly secured) rlogin rcp rsh... you want to rename those and replace them with the non secure versions, there are other ways, but this saves a bunch of hassle. 6. pop into /etc/pam.d and adjust the rlogin rcp rsh.. (this may not be needed in some cases). 7. add in a + + in the .rhosts file of each cluster user.

after you have pulled your hair out decyphering my glossed over instructions, you should be able to type: rsh node002 and be at the prompt for node002 with no password asked, and no silly kerberose failed: trying /usr/bin/rsh message given.

At this point then you can configure LAM (you may nee to download it and get it installed on your boxes)

basically it needs an arbitrary file Mynodes.txt that will contain the list of nodes you wish to launch. you type in lamboot Mynodes.txt and then it will kick back some silly error 99% of the time because something small was forgotten. you fight through those errors until it finally gives you a sucess.

Now your golden, then its just a matter of figuring out how to compile and run MPI programs with the mpiCC and mpirun. But if you got through the first gloss over then the rest is a snap.

Remember if these machines see the outside world they are naked, defenseless, and totally exploitable..

Be aware that these instructions can cause all sorts of havok and any reasonable person would just hire an expert.

Honestly I hope that this gives you a starting point. You'll still need a bunch of time with google.

GOOD LUCK!!

Storm

OpenMosix LiveCD

[dargaud]

First, here are some notes about the first cluster I build [gdargaud.net] 3 years ago, and most of it is still relevant.

This being said, for an instant trial, there are some OpenMosix LiveCDs, such as Quantian or other variants of Knoppix. Put the Quantian DVD in the 1st PC, boot, enable the remote boot option, boot the other computers over the network. Here: you have an operational cluster.

I think there may also be Rocks LiceCDs but haven't tried them. And remember your electricity bill when playing with clusters !

my first cluster setup

[Wornstrom]

For my first diskless cluster, I used the warewulf cluster solution [warewulf-cluster.org] to see one up and running. Then, I wiped the master node's disk and built one with the openmosix kernel patch etc, and used the Linux terminal services project [ltsp.org] which was really cool. the ltsp stuff made the node filesystem stuff easy to build onto. I am waiting for the openmosix team to finish up work and release the userland tools for the 2.6 kernel for my next build. here is a good how-to on LTSP+Openmosix - http://ltsp.org/contrib/ltsp-om5r3c.html [ltsp.org]

Once you get that going, you might look at PVFS2 [pvfs.org] Parallel Virtual File System. "PVFS2 stripes file data across multiple disks in different nodes in a cluster. By spreading out file data in this manner, larger files can be created, potential bandwidth is increased, and network bottlenecks are minimized."

Good Luck!!