How To Properly Archive Data On Disc Media 120
An anonymous reader writes "Patrick McFarland, the well-known Free Software Magazine author, goes into great detail on CD/DVD media over at the Ad Terras Per Aspera site. McFarland covers the history of the media, from CDs through recordable DVDs, explaining the various formats and their strengths and drawbacks. The heart of the article is an essay on the DVD-R vs. DVD+R recording standards, leading to McFarland's recommendation for which media he buys for archival storage. Spoiler: it's Taiyo Yuden DVD+R all the way. From the article: 'Unlike pressed CDs/DVDs, burnt CDs/DVDs can eventually fade, due to five things that affect the quality of CD media: sealing method, reflective layer, organic dye makeup, where it was manufactured, and your storage practices (please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media).'"
Dupe... (Score:5, Informative)
Re: (Score:1)
Re: (Score:3, Interesting)
Re: (Score:3, Interesting)
I'm not too happy with the way the author recommends one particular brand over others like that without any hard data to back it up.
To be fair, whilst I'm not claiming that I've never come across criticism of Taiyo Yuden, they seem to be consistently ranked #1 in reliability and quality. In fact, come to think of it I can't recall seeing any reviews where their media (overall) weren't the top rated.
I believe that Verbatim (owned by Mitsubishi) are also very highly rated (except for a brief period in 2002 when they switched to a far less reputable media supplier). More info in this article. [digitalfaq.com]
Bear in mind that a *large* number of majo
Re: (Score:1)
It's worth noting that this article is specifically talking about data archiving (obviously), where the +R format does offer advantages with error correction/handling. One size doesn't fit all for media however, and I still recommend using DVD-R for video DVD authoring as it has a much high
Re: (Score:2)
I saw some figures showing the percentage of "modern" DVD players and the difference in the number of models that could handle DVD+R and those that could handle DVD-R but not DVD+R was only something like 2% to 4%.
Re: (Score:2)
It's common knowledge among the CD recording crowd that Taiyo Yuden - a Japanese brand - is the top maker of media. I always buy Fujitsu brand DVDs (although not CDs, because CDs are less critical) because most of their media is made by TY. It's an issue with the proper spreading of the dyes, apparently. The Japanese have the quality control standards that the Taiwanese simply don't have (except in cases where a Taiwanese company is using Japanese equipment and methods, which does occur.)
Check out the forum
Re: (Score:2)
Re: (Score:2)
I think the consensus of burner experts is that NEC is way better than LiteOn. I see NEC being sold by the local computer shops - but not LiteOn. With the essentially zero difference in price, NEC is the way to go.
Re: (Score:2)
This isn't a dupe. (Score:3, Funny)
Re: (Score:1, Funny)
Oh how I miss 2006, the internet has really gone downhill since then.
Re: (Score:1)
You must be new here...they usually wait less than 4 days to dupe it.
M*Farland? (Score:5, Interesting)
It's not like we haven't seen that before ( Roland P* )
Patrick McFarland? (Score:5, Informative)
Anyway, just seemed fishy to me. That's my $0.02.
Re: (Score:1)
hmm... (Score:2)
Re: (Score:1)
Re: (Score:1, Informative)
Re: (Score:2)
Why bother with optical? (Score:3, Interesting)
Re: (Score:2)
Re: (Score:1)
I just bought two 500 GB drives ($139 each) last night for the very same backup strategy that you suggest. I
Re: (Score:3, Informative)
Re:Why bother with optical? (Score:5, Insightful)
Yes, you'd save a few bucks initially by using optical media. But then you have to split all your data in evenly sized 4.7GB archives or such. Label them all manually. Waste hours swapping hundreds of discs by hand. Catalog all these discs (number them, keep a database or something0. And they take LOTS of physical place to store.
I got tired of looking for a specific DVD. After an hour of flipping thru pages of those (expensive and large) CD wallets and not finding it, I gave in, and bought several TB worth of HD space. Now if I want a movie, it's there, listed alphabetically and all. Jewel cases suck too -- too brittle, wastes space too, and a waste of money.
HDs have a very high density (the new 1TB drives will store more than a spindle of 200 DVDs), requires no storage cases, no constant media swapping, no splitting to fit the size of media, etc. It's all-around better! Restoring stuff is almost instantaneous. You never have to look for a specific disc. Transfer speeds are great. Everything is sorted alphabetically inside folders/directories... What more could you ask for?
Want to make a new copy - to another format, or just onto newer media? It takes what, 15 seconds to start a copy job using HDs? With optical media, you'll be swapping discs by hand for months.
Plus, HDs are not read only -- you can make monthly backups on them no problem (full or differential). Using optical media this is a pain. The cheap discs are write-once, and even if the rewritable discs were cheap, it still takes a while to erase them all manually.
Backup/sync jobs can be totally automated using HDs. Want to do daily differential backups? Schedule it once, never have to bother ever again. With optical discs you'd be swapping, labeling and cataloging media 365 days a year... What a pain.
Long story short, I might as well backup all my storage on 1.44MB floppies instead of optical discs. It'll take hundreds or thousands of them, and you'll spend countless hours swapping media and splitting stuff to fit the media size. NO THANKS!
Re: (Score:1)
Re: (Score:2)
HDDs keep getting cheaper in terms of dollars per byte stored. They are also getting faster, smaller and more reliable. Why bother with anything else?
Re: (Score:2)
Bought two 250GB disks (Cdn$70 each on sale), and two USB enclosures (Cdn$27 each).
Works out of the box with Linux.
A cron job does an incremental dump to BOTH disks once a day. Once a week, a level 0 dump is performed and several versions of that are kept.
Details here:
- Setting up a hard disk USB 2.0 enclosure for backup under Linux [baheyeldin.com].
- Ubuntu Linux backup of a laptop using a USB enclosure and the dump utility [baheyeldin.com]
Drawbacks?
1. The silent enclosures have no fans, and are
Re: (Score:1, Insightful)
Oh, so your time spent burning DVDs is worth nothing eh? Let me give you a call sometime so you can back all my shit up for free.
I have a cron job that executes at 4am that copies everything to a USB external hard disk. I think it took a
But how reliable are HDDs? (Score:3, Interesting)
Say you use them for archiving. It is better to go the NAS route and have them always on, or powered down when not in use? How about a USB enclosure only connected when in use? How reliable is an almost-new used-it-once HDD that has been sitting on a shelf for five years?
Re: (Score:2)
Re: (Score:2)
Still, there are no hard numbers on reliability. RAID is not a backup solution, it's about uptime and availability which is pointless in this context. It's no good having two identical disks if they both get corrupt or the heads seize up after a few years.
One argument against alawys on is that it is more prone to electri
Re: (Score:2)
Ideally one would power them off and even remove them to an offsite backup regularly, but the convenience of not having to go into the basement/computer room every day makes it more convenient to keep them on always.
See my comment here [slashdot.org] for more details.
Re: (Score:3, Interesting)
What's your time worth? As someone said, backing up 100GB of data - something I do periodically and I do it on DVD - takes time. I back up over 200GB of data off my machine to DVD - it takes me most of a day of personal attention. Backing that up to an external hard drive would take much less time and require nothing more than starting the transfer (depending on how the files are being selected of course).
That said, DVD (or tape, of course) is better for offsite archival storage than disk in many, if not al
Re: (Score:3, Interesting)
From personal experience optical discs might need one more step though: to verify that the write was ok. I don't have any optical discs that have failed me yet after I started to check writes.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
data on it. (Granted, there may be ways to recover some or most of the data, unless the drive is just utterly
destroyed in a fire or something, and if it was a fire, your whole box of DVDs would probably be wiped out
too). Also, with DVDs or CDs, one buys incrementally to archive; one could put
some money in an interest bearing account for instance, though it's a trivial amount. Finally, there's a
ps
Re:Why bother with optical? (Score:4, Insightful)
An archive is something that is stored in a safe location for possible use later, but also to save for posterity.
A backup is used to keep current data in two locations in case one set of data is lost.
Hard drives are fine for backups. For archives you want write-once media that can't be easily (or even possibly) erased.
Two different solutions to two different problems.
Re: (Score:3, Funny)
Re:Why bother with optical? (Score:4, Interesting)
I typically create data DVDs with 30% redundancy information (though smaller percentages are probably more than adequate) with par2create, and store those par2 files on the same DVD. That way I survive the little scratches and can recreate the data.
Isn't there already some redundancy? (Score:2)
Not saying that using PAR2 to create some additional redundancy is a bad idea, but I don't think that one-bit errors are totally disastrous automatically, if you've burned your data in a normal format.
Not sure what UDF does, though.
Although it's not as if PAR2 is a particularly exotic format, just in case it drops out of common use in the future, I think I'd still put a plaintext copy of the source code for the reassembler on every
Re: (Score:2)
The problem with ECC at the disk level is more of a user interface problem. When a disk starts failing (and the ECC is kicking in), you don't get notified. At least, not until it's too late and the ECC can no longer fix the errors. Attentive users may notice that the disk is having trouble seeking / reading but most users won't.
When you add PAR2 data to a disk, you create a window of time where you can recover the data after the disk shows obvious ECC errors.
Re: (Score:1)
he's right (Score:1)
you see, SOMEthing corrupted some jpgs at some point between 5 years ago and now. the fact that i was backing up my entire digital camera photo repository to another harddrive didn't help. the corrupted jpgs were backed up as well.
so you see, having an optical burn does indeed help. his point about the data not being overwriteable is completely valid. another harddrive isn't good enough.
Optical doesn't get fried by power surges. (Score:2)
Ditto with mechanical shock -- a DVD will survive a lot rougher handling than a harddrive will, even if the latter's heads are parked.
There are always trade-offs.
Re: (Score:2)
" I've never heard of a CD or DVD drive's laser suddenly burning holes in the disc.
I've had cd drives that ate the cd, and one time ejected a cd at high speed. Think frisbee, not coaster. When a disk fails at 52x, your PC will sound like an unbalanced washig machine.
Ditto with mechanical shock -- a DVD will survive a lot rougher handling than a harddrive will, even if the latter's heads are parked.
Drive over a hard drive, then put it back in the machine - it'll work. Just drop your 100-spindle sta
Re: (Score:1)
Re: (Score:2)
Which leads to... (Score:2, Insightful)
Is there a point to digitizing human cultural pieces that has survived for 10,000 years onto media that will fail in 10?
Re: (Score:2)
Re: (Score:3, Insightful)
That is certainly not true, and even worse digital is less resiliant to errors than analog formats.
Re: (Score:2)
That is certainly not true, and even worse digital is less resiliant to errors than analog formats.
If there are errors, I don't recall them ever happening in the copying, they happen in storage. And while single bit errors can happen, with checksums I can verify that I have files that are closing in on 20 years old that are perfectly preserved. With PAR files I could even re
Re: (Score:1)
However.... There are plety of cultural pieces that have survived in harsh environements, wars, etc. and are still in relatively good condition. Put a cd outside for 1 year and its fubar. Thats the point I was trying to bring across.
We may be able to make perfect copies indefinetly, but at some point in the future, someone, somewhere, will have the great idea of digitizing art, and destroying the original, then leave the only copy of Mona Lisa out
Re: (Score:2)
Sure, my HDD might die someday, or the DVDs rot, or whatever... But then I just go re-download all my critical files.
Looks like I have about 300 backups distributed around the globe of this one chunk of stuff! =D
Re: (Score:1)
Lucky77
Re: (Score:1)
Or, if you want to do away with the "generations" altogether, engrave the digital data into a rock — with suitable instructions on how to decode, of course. That way your data becomes immortal, more or less. On the other hand, you would probably need a mountain-sized rock for the average TIFF.
Re: (Score:2)
Re: (Score:2)
A more complete anwer to your question: Yes, it is worth it. Existing peices of history may have "survived" thousands of years, however I'm not aware of anything that has survived that time completely intact. And despite all the modern preservative techniques at our disposal we still cannot completely prevent the degradation of physical artifacts. Digitised content on the other hand can be copied infinitely and perfectly preserved
Re: (Score:1)
Formats will change, but paper will always be paper. Marble will always be marble. And art (in all its forms) will always be art.
Re: (Score:2)
Re: (Score:2)
If an artifact has existed for 10,000 years, only a fool would suggest that we destroy it and only retain copies that are on an untested medium. However, if some new medium makes the data more convenient and useful, then there's no downside, really, to making it available in both -- keeping the old format and also copying it into a modern electronic one.
Just as a trivial personal example, I have a lot of photographic slides and negatives. Some of them (the on
Re: (Score:1)
Remember "Raiders of The Lost Ark"? How at the end, some old guy hides the Ark in a government warehouse? You might write your important cultural stuff in stone, but if nobody can find it, it might as well not exist. Another example: the beard of the Sphynx is sitting in a back room somewhere in the British Museum. But nobody knows it's there, so who
Re: (Score:1)
Re: (Score:2)
Moses' stone tablets (Score:2)
whilst the data density of stone tablets is quite low, if carefully stored they're pretty much readable indefinitely!
Re: (Score:1)
Disks BAH! (Score:2, Funny)
Note: With I can gather from the name of the site, it appears they might also sell hats, but I couldn't find a link.
Dude - you need to update! (Score:2)
Check this out for all the tactile comfort of cylinders, with the convenience of digital - paper tape [baudot.net] is where it is at!
Re: (Score:1)
What ever for? I'm futureproof [tburke.net]
TNX Z
Redundant Archive of Inexpensive Discs (Score:2)
That way, any "spot defects" will be very unlikely to hit the same data on every copy. Making the whole redundant backup set last many times longer.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
This means on a 700 MB CD, you have ~840MB of space. Audio CDs use this extra space because data corruption isn't too significa
Re: (Score:2)
And you can't use the "extra" bytes above 2048 per frame for data, because you need the filesystem to do what it does. If you encoded in CDDA, which is no filesystem/overhead, you start losing data immediately on different players tha
Incorrect. (Score:2)
Also, the error correction/detection is not dependent on having any particular file system on the medium. Any data CD (regardless of file system) has the dat
Re: (Score:2)
But that's not error correction for the data decay we
Re: (Score:2)
The level at which error correction on a CD/DVD occurs is not the file system. I can do the equivalent of dd=/dev/HUGE_FILE of=/dev/cdwriter and error correction will still be applied. Of course you may have a different notion of what a file system is, but generally a storage layer isn't considered a file system unless you actually use it for organizing files... which the "sect
Good experience with DVD-Rs myself (Score:4, Informative)
Some media have been observed to degrade fairly rapidly, others are quite stable. About a year ago, and again recently, I scanned a number of relatively old DVD-R discs [backups, uh, owned by a friend] burned from fall of 2002 on. You can see my post here:
http://club.cdfreaks.com/showpost.php?p=1733269&p
Funny thing is that most of the discs I used were of a brand widely lambasted as "cheap ____" and I was told that they wouldn't last six months. Curiously enough you can see that the cheaper "Princo" media has held up better than the "gold standard of the day" Riteks [although both are much better than some]. You can also see that one of older discs was scanned recently, and more than a year ago. It shows almost no degradation during that time [and what it does could easily be attributed to the aging scanning drive].
The CDFreaks forum has a lot more scans, including of older media. If you've got some discs and are worried about their aging stability, here's a good place to start:
http://club.cdfreaks.com/forumdisplay.php?f=33 [cdfreaks.com]
Media testing in Linux? (Score:1)
haha (Score:1)
good old days (Score:1)
Now people complain if you can't burn faster than 24x. Ask them what color their media was and you'll get something like, "Its magenta with Sony written across it." Peop
the whole premise os wrong (Score:1)
He told me in many ways, why any kind of disk archive was better than tape. I've thought about that conversation many times.
My conclusion FWIW, is the same as everyone knew half a century ago. Accessibility-density-longevity... pick = two.
Fails to mention MO (Score:3, Informative)
- http://md5.ca/~pavel/md.jpg [md5.ca]
- http://tinyurl.com/2cu7zv [tinyurl.com]
MO drives are a bit costly, but if you have important media its worth it. Besides cool look for neo's warez stash in Simulcara book. Quoted guaranteed archival time is over 40 years in most cases, and they continually improving the technology, compared to driving the costs down of the generic blank media market of CDs/DVDs.
Re: (Score:2)
"Magneto-Optical media is used by medical facilities where archival time length is paramount."
My last magnto-optical media died from the click of death, you ignorant clod.
Re: (Score:1)
Other tips: (Score:2, Funny)
- Use acid free CD-markers when marking
- Do not use as a coaster like your America Online CDs
- Do not play Frisbee outside with your CDs (Exposes to direct sunlight)
- Chewing reduces durability
Obsolete conculsions (Score:1)
We're to use Taiyo Yuden's "Super Cyanine" "stabilized" Cyanine dye, because for a short time, it was the best dye. But then the article goes on to admit that TDK' "metal-stabilized Cyanine" is rated for the same shelf life. Then it goes on to admit that the "Metal Azo dye" I see listed on the brands I've been buying is rated for a longer shelf life than either of the previously mentioned ones,
DVD-RAM would be the way to go if 12x media (Score:1)
Use dvdistaster (Score:4, Interesting)
DVD Writable sucks badly as archival technology. (Score:4, Interesting)
My conclusion is that either people do not care about their archived data or that the number of, e.g., lost baby- and wedding-photographs is not high enough yet.
Re: (Score:2)
Re: (Score:2, Informative)
Re: (Score:2)
Re: (Score:2)
Link e.g. here: http://www.fujitsu.com/global/services/computing/
Re:DVD Writable sucks badly as archival technology (Score:2)
Just use a hard drive (Score:1)
and after diagnosing the problems... (Score:2)
For workstation-size data sets, I use a drive mirror for short-term and DVD+R for long term... and looking forward to terabyte recordable in 5.25" form factor which should be available in a few years.
DVD Ram? (Score:1)
two words: fired clay (Score:2)
I can't reply to this april fools stuff any more, my head hurts.