Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?

Slashdot videos: Now with more Slashdot!

  • View

  • Discuss

  • Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

User Journal

Errtu76's Journal: Clustered file systems

Journal by Errtu76

We're currently experimenting with clustered file systems. In the running to become our selling products are GlusterFS and Hadoop. The first because of its flexibility and easiness of setting it all up, the latter for the sheer fact that large companies are already using it and on paper it looks very slick. Hadoop only has the drawback of requiring java which in turn means the relatively cheap nodes suddenly become more expensive since we have to buy more ram.

GlusterFS should be able to support striping over AFR (automatic file replication) and thus supply a good mix between performance and reliability but so far i'm unable to produce a working configuration for this. The only setup i'm using right now is a 20-node cluster which contains 10 AFR pairs and are clustered with Unify. Even over GigE this performs quite well. I don't see any performance increase with the various optimizers yet. I have tried all but the 'boost' optimizer, so that's still worth a shot. Also some days ago 1.3.10 was released which fixes a hang bug when pulling out an AFR node. I'm not sure if this will fix the overwrite bug then, as i had to use the latest mainline 2.5 version for that fix.

Hadoop will be tested next week if all goes well. It will be just a matter of taking out the 256M RAM modules from 10 nodes and placing them in the other half, so java should have enough. Why one would develop such a system in java is beyond me, but then again, i'm not a developer so they must have had good reasons to do so.

Why am i typing all this? I don't know. Go see icanhascheezburger.com if reading this made you feel sad.

This discussion has been archived. No new comments can be posted.

Clustered file systems

Comments Filter:

"Well I don't see why I have to make one man miserable when I can make so many men happy." -- Ellyn Mustard, about marriage