Bulk Data Storage For The Common Man?

Vigyaan writes "Lately, I have been looking into different bulk data storage options available to a common man. My work depends on generating, storing and analyzing a large amount of data -- averaging about 1 TB per month. I would like to have a storage system which is automated, fast, reliable and most importantly does not cost the price of an eye. Right now, I have a 4 node Linux cluster with 10 large hard disks (total capacity 1.6 TB); data storage roughly costs about $0.60/GB (excluding the cost of PC hardware). But long term storage is painful -- DVDs cost about $0.10-$0.15/GB but takes too much human time and leaving data on hard disks makes me nervous because of possible failures. RAID is a possibility, but it increases the cost significantly. I was wondering, if Slashdot readers have any recommendations for a cheap automated way to store and retrieve data."

Finally a use for my 1GB Gmail invites...

[anakin357]

I'll send you a couple.

I have one

[Anonymous Coward]

Floppy disks.

good luck

[Madcapjack]

PRINTSCREEN should do the trick.

Re:I have one

[Anonymous Coward]

Punchcards

1TB a month?!?

[stinkydog]

Short of launching his own space probe, the only way for this guy to consume a TB a month of storage is a serious porn habit. Just post your 'content' on Edonkey and it will be available when you 'need' it. You likely only watch them once anyway.

SD

Bulk storage?

[mikers]

I got a couple of drawers of old floppy disks. $10 takes 'em. Plenty of bulk.

The Sony "lifetime" warranty may still be good on them too!

Oh, I see...

[spinkham]

You want it fast, cheap, reliable, easy, and now, eh? Good luck with that.... Sounds like a request from the PHB...

Re:age old problem...

[D-Cypell]

my data is worth over $100,000 a month

This 'data' doesnt happen to be a large collection of email addresses does it?

spongedrive is best

[cubyrop]

i am responsible for providing storage solutions for a mid-sized content creation company which, through version archiving, accumulates near 1-200 GB per day. they require access to their media backups on a rolling basis, so tapes are not an option.

i have found that a Teutonium cluster of 6.5 TB Spongedrives (either Cray or SecreTech are fine) fits the bill nicely. housed in a 15-unit rack server, the amoeba-shaped drives utilize BioLas technology to store data on 6-dimensional Moebius Cilia for a slick seek time of 0.00 ms.

a cluster costs about $45,000 USD but the price should come down in 2004 Q4 when SecreTech launches their new 40-platter blackholium SCSI's.

safest way

[nazsco]

echo 1tb.txt > /dev/lprn0

Re:1TB a month?!?

[grasshoppa]

Using this method, I have achieved my life long dream of tapeless ( well, everything-less ) backups.

I simply make a tar.bz2 file with all my important files, filter it through gpg, then post it on edonkey, usually titled, "Olsen twins getting it on", and then usually the date.

Viola, instant backup that is available to me whereever I may go.

Easy

[Pedrito]

I use bioneural gel packs at a cost of $0.04 per teraquad. What is this hard drive of which you speak?

Hijack Cassini

[Anonymous Coward]

... and program it as a repeater.

It's about 90 minutes away, so at 250 Kbps that's over one terabit in storage on the way out there, and another terabit on the way back.

Worst-case access latency is about three hours, though. Maybe the hard disks are a better idea.

If you send your probe^H^H^H^H^H repeater to Alpha Centauri, you'll get more than 20,000 times the storage capacity.

Re:1TB a month?!?

[gl4ss]

** You have obviously never heard of fMRI studies, have you?**

oh shit! I totally missed the part of the history where FMRI scanners came commonplace for men.

oh wait the whole ask slashdot blurb is twisted, the headline implies asking for datastorage possibilities for the common man - yet one of the first things mentioned that he needs it for his special job that generates tb's of data per month. by that definition he is not a common man, except that he hopes to have a miracle solution - that is quite common.

still, a common man would choose whatever possibility gave the cheapest price per gb(probably harddrive). with dvd-r's he would end up burning multiple dvd-r's per day and it's kind of implied that the data would need to be retrievable so he would have to burn the same disc multiple times, even then it wouldn't be a sure thing.

his needs are quite bigh though still, big enough to warrant for professional help since his likely going to be spending quite a bit of money on the thing.

Re:1TB a month?!?

[Zone-MR]

Oh, so that explains why that "Olsen Twins Getting it on - 12 Mar 2003.avi" file I downloaded last week contained a zipped tar archive full of boring spreadsheets and a lot of donkey porn.

Re:I have one

[Chess_the_cat]

Just do what I do: memorize it all. No need for binders, cards, floppies, HD, DVD.

Re:good luck

[mlk]

please dont joke about that, not too long ago I received an email, please help me open this word document. Complete with a BMP, natrally I assumed this was the error message. Nope, it was a "print screen" of its Icon. And yes, while I had said picture up, the user did try double clicking.

Poor luser.

Only on slashdot

[gexen]

Only on Slashdot would they start talking about huge storage arrays and title it "for the common man"

Re:options options, what is your time and data wor

[Anonymous Coward]

"you need to either a) get a grant to fund your work, b) hire somebody to swap DVD's for you or b) seriously rethink your data generation."

I choose b ... HEY! That's a trick question!

Meh.

Re:Use those HDDs!

[Cecil]

I'm just curious, do you have any idea how much data 1 Terabyte is? Are you suggesting that he PRINT it?

Let's say for the sake of argument that all 256 bytes can be printed as a visibly distinguishable character, or that he's got 1TB of plaintext. Also assume you can fit 10,000 characters on a 8 1/2 by 11 page.

You can fit 10^4 bytes per page, and you need to print 10^12 bytes (I know, it's actually 2^120, but that needlessly complicates the math, so shush)

That means you will need 10^12 bytes / 10^4 bytes/page = 10^8 pages.

One hundred million pages. Assuming he has a good laser printer with infinite toner, let's say he can print 60 ppm or one page per second. It would take one hundred million seconds to print the data, which is 1157 days, or a little over 3 years.

Given that he generates 1TB per month, I think this backup plan would probably become the top agenda item of most of the anti-deforestation groups out there.

Re:Finally a use for my 1GB Gmail invites...

[Anonymous Coward]

That's [e-mail] ? Just wanted to make sure I got it right. [e-mail] . I'm guessing you'd want to have [e-mail] as your e-mail address? Invitation is on its way!

Re:Finally a use for my 1GB Gmail invites...

[EvilTwinSkippy]

Nah, just tarball your backup into 1 or 2 GB file sizes, name it "PR0N XXX TEEN SEX DONKEY LOVE - MILITANT ISLAMIC BUKAKKE KITTEN.MPG.AVI.WMV" and share is on Gnutella.

Re:1TB a month?!?

[EvilTwinSkippy]

Well if you are the one behind all the "Islamic Militant Bukakke Kitten" porn, you are one sick bastard.

And you really need to fire your accountant. Your Caymon Island bank account was overdrawn twice in a month.

Re:good luck

[schtum]

Translation:

Please don't joke about that. Not too long ago, I received an email asking for help opening a Word document. Attached was a bitmap image which I naturally assumed was an error message. Instead, it was a screen-capture of the document's icon! The user was double-clicking on the image!

So I shot her.

Re:Hard disks

[Naffer]

I think you're absolutly right.
Tapes or cheap.

Re:Wirewire drives?

[NanoGator]

"here are no external firewire drives. There are (to my knowledge) no Firewire drives at all.
People take standard ATA/IDE drives and use an ATA/Firewire bridge to connect them up externally and bypass the extremely limited cable length of ATA."

Well that totally blows my point out of the water!

Re:Xraid

[Lukey Boy]

I do believe that this is the first time I've ever seen the word Apple used in the same sentence as "reasonably priced".