Become a fan of Slashdot on Facebook


Forgot your password?
DEAL: For $25 - Add A Second Phone Number To Your Smartphone for life! Use promo code SLASHDOT25. Also, Slashdot's Facebook page has a chat bot now. Message it for stories and more. Check out the new SourceForge HTML5 Internet speed test! ×
User Journal

Journal Errtu76's Journal: Clustered file systems

We're currently experimenting with clustered file systems. In the running to become our selling products are GlusterFS and Hadoop. The first because of its flexibility and easiness of setting it all up, the latter for the sheer fact that large companies are already using it and on paper it looks very slick. Hadoop only has the drawback of requiring java which in turn means the relatively cheap nodes suddenly become more expensive since we have to buy more ram.

GlusterFS should be able to support striping over AFR (automatic file replication) and thus supply a good mix between performance and reliability but so far i'm unable to produce a working configuration for this. The only setup i'm using right now is a 20-node cluster which contains 10 AFR pairs and are clustered with Unify. Even over GigE this performs quite well. I don't see any performance increase with the various optimizers yet. I have tried all but the 'boost' optimizer, so that's still worth a shot. Also some days ago 1.3.10 was released which fixes a hang bug when pulling out an AFR node. I'm not sure if this will fix the overwrite bug then, as i had to use the latest mainline 2.5 version for that fix.

Hadoop will be tested next week if all goes well. It will be just a matter of taking out the 256M RAM modules from 10 nodes and placing them in the other half, so java should have enough. Why one would develop such a system in java is beyond me, but then again, i'm not a developer so they must have had good reasons to do so.

Why am i typing all this? I don't know. Go see if reading this made you feel sad.

This discussion has been archived. No new comments can be posted.

Clustered file systems

Comments Filter:

"They that can give up essential liberty to obtain a little temporary saftey deserve neither liberty not saftey." -- Benjamin Franklin, 1759