Bit Rot Stalks Your Digital Keepsakes 535
axlrosen writes "The NYTimes has an article about the problems of digital archiving. How many of your digital memories will still be around 50 years from now, considering lost disks, incompatible formats, hard drive crashes, fading CD-Rs, etc.? Unfortunately Peter Briggs' solution won't work for most of us. The only real way to make sure that your grandkids get to see your digital photos is to make real photographic prints from them. (When I bought my Mom a digital camera I installed Picasa for her, and made sure she knows to order real prints of all the pictures she wants to survive through the ages...)"
Tell me about it (Score:4, Funny)
Re:Tell me about it (Score:5, Informative)
A better read though, is this [about.com] which is an article about who to best go about long term storage on CDRs.
It includes the tip, amongst a load of others, that the top of CDR's is far more fragile and needs to be treated with great care.
Re:Tell me about it (Score:4, Insightful)
If you drop and scratch a DVD, you could lose ten thousand photos.
If you drop a photo album, you'll scratch a picture or two.
For anything I want to keep, I'll stick to a 35mm camera. For ebay or computer stuff, I'll use a digital camera.
Re:Tell me about it (Score:3, Funny)
Also, if you carry your ten thousand pictures around in a shoe box (a BIG shoebox), you will scratch a lot more pictures that way, even if you don't drop it. The shoebox is a better analogy than an album than dropping a bare DVD. If the DVD was inserted in a jewel case before it was dropped, it probably wouldn't scratch the DVD. A DVD in a jewel case is a better analogy to pictur
Re:Tell me about it (Score:4, Insightful)
Re:Tell me about it (Score:3, Insightful)
Uh-oh, I think they heard me -- some men in white lab coats are knocking at my door. Gotta go!
Re:Tell me about it (Score:5, Insightful)
This guy's advice is not smart. Bascially he's saying "take your perfect copy that might die at some point and replace it with an imperfect copy that is guaranteed to deteriorate with age." Heck, I'll just laser print all my documents for backup as well. We all know there's no way they could possibly be lost then. We all know going analog is much safer than backing up and refreshing the data on new media periodically because all those prints of movies, music and documents from 75 years ago look and sound so damn good.
I'll take my chances with backing up and copying data periodically over my skills as a museum currator any day.
TW
Re:Tell me about it (Score:3, Insightful)
Many cameras are taking pictures using camera raw - camera raw pictures can really only be read in a few programs right now - one of them is photoshop.
Take pictures made on computers 20 years ago - can you read those pictures easily right now? You'd probably have a hard time reading pictures made with graphics app that dec, quantel etc made back then.
Re:Tell me about it (Score:3, Insightful)
You can restore (or pay somebody to restore) a badly deteriorated, hundred-year-old photographic print, and get remarkable results. Imagine trying that with any digital media.
And you say you don't want to be a museum curator, but you're choosing the option that will require exactly that. Digital image archival will require meticulously cataloging, inspecting, and duplicating your media library. If you make prints, you can stick them in a shoe box and
Re:Tell me about it (Score:3, Insightful)
I'm not a photo geek. The pictures I take are snapshots, not art. When I print them out they end up in a frame from Target or a photo album from Wal-Mart or sometimes a shoe box on my shelf. These pictures will not last and I'm unwilling to go to the effort of to print them and store them in such a way that they do last.
I am a tech geek though. I have more than a dozen functioning and used
Re:Tell me about it (Score:3, Informative)
Amen to that. You know this same damned topic comes up on slashdot about every 9 months. And every time, I interject the same thing:
The best method of archival storage of color images is an archival quality CD-R!
The CD-R takes up so much less space than a rack of kodachrome slides (the only color archival quality film) and is orders of magnitude less expensive and an order of magnitude higher quality than a box of pigmented ink printed pictures, that it wins hands down.
Re:Tell me about it (Score:3, Interesting)
"In the digital world, we don't need back-ups, because a digital copy never wears out. It is timeless." -Jack Valenti, former head of the MPAA, 2002 interview with Harvard Political Review's Derek Slater
Re:Tell me about it (Score:5, Insightful)
Re:Tell me about it (Score:5, Funny)
Re:Tell me about it (Score:3, Insightful)
Re:Tell me about it (Score:3, Funny)
I believe you forgot pr0n
Re:Tell me about it (Score:3, Insightful)
Multimedia grows significantly : NTSC PVR requirements now are nothing compared to HDTV requirements, forget whatever we dream up in the next 20 years.
Games take as much storage as a couple of hours of motion video.
Microso
Every 2-3 years (Score:4, Insightful)
Re:Every 2-3 years (Score:5, Insightful)
Or rather, dispense with the concept of permanent media altogether. I realised a few years ago that the only sane way to protect my data was to have it all online all the time. I store my data on redundant arrays of disks in two geographical locations (my house and my parents' house, synced nightly via rsync). This is IMHO a far better solution than backing up to tape or CD/DVD. LVM makes the process of moving the data to bigger disks trivial. Where it falls down is for really large volumes of data. Places like CERN that generate terabytes of data per day are going to struggle in the not too distant future. Archived data will become a real problem (even more than it is now).
I dunno (Score:5, Funny)
Re:I dunno (Score:5, Funny)
Just protect your computer by placing a firewall around it
There's still a single point of failure (Score:3, Interesting)
What if somebody hacks your primary machine and erases your data? This would propagate to your backup server as well. I see at least two solutions to this: 1) make a WORM copy every so often and/or 2) write to the backup server is a journaled manner so that ol
Re:There's still a single point of failure (Score:3, Interesting)
The syncs are delayed, so I have an overnight sync to a local disk in my main machine, weekly backups offsite, and 4 weekly backups from that to another offsite machine. Thus I have 28 days in which to spot the deleted data and restore from backup (actually, I don't need to spot it manually -- AIDE [cs.tut.fi] tells me when a file disappears from my machine). Eventually, I'll get around to implementin
Watch out for mistakes (Score:5, Insightful)
Do you run rsync with --delete? If not, how do you deal with moved files? If so, how do you deal with accidental deletion?
I grant that you've solved the decaying media problem, but I've lost more data to screwups than to bitrot.
Re:Watch out for mistakes (Score:5, Informative)
I keep several months worth of point-in-time "copies" of my home dirs, mail, /etc, and other stuff online and available on separate hardware.
Re:Watch out for mistakes (Score:3, Interesting)
I'm not the person to whom this question was written, but I'll tell you my solution:
My DB dumps are my biggest chunk of data. I dump each table (in each schema in each DB) to a separate stream and break the stream up into chunks of a specific size (configurable per stream). For each chunk, I maintain an md5. Every day, for every chunk, I compare the md5 against the md5 for the same
Re:Every 2-3 years (Score:5, Interesting)
A daily backup of important files (and stuff that is changed daily) goes to all machines in one shot at ~6am.
A weekly backup of EVERYTHING goes to three different machines every Sunday at ~5am.
Now, I realize that all three could be screwed simulataneously but at least I know that TWO of those machines have automated backup to CDRW daily.
Yeah, it's paranoid, it's redundant, but it's my data and it's important to me. If I lost my 2300 pictures I'd be lost.
Re:Every 2-3 years (Score:3, Insightful)
Yes, but photos aren't protected from fire, flood or other natural disasters. The point of the article was to make a system that was at least as good at long-term archiving as print photos.
Re:Every 2-3 years (Score:5, Funny)
Re:Every 2-3 years (Score:5, Funny)
Just tell them not to worry, it's awl write.
Umm (Score:2, Insightful)
Re:Umm (Score:3, Insightful)
Re:Umm (Score:3, Informative)
Nobody's too sure how long inkjet printouts will last. My own printer's inks and paper are supposedly safe for a century (according to accelerated ageing tests using ultra-violet lamps, or something similar), but I'll still be keeping all the original JPEGs, regularly backed up on to some lowest-common-denominator medium (currently CDRs).
Professional digital photo prints are likely
Re:Umm (Score:5, Interesting)
Re:Umm (Score:3, Funny)
Re:Umm (Score:3, Interesting)
Standard colour prints are made with organic dyes. Those fade in time.
Black and white photos are silver
Those don't fade but other factors like the underlying paper turning yellow or the underlying film being cellulose nitrate a close relative of nitro cellulose (AKA gun cotton) causes it to disintegrate.
For the chemical holy grail, (and this goes back to my knowledge gained in the 80s and early 90s)you're looking at Kod
But very differently.. (Score:5, Insightful)
I have news for you (Score:4, Funny)
Re:Umm (Score:5, Informative)
Properly cared for, black and white negatives will keep for a very long time. Nobody knows exactly how long "a long time" is, but negatives from the turn of the last century are still perfectly viable.
Colour materials are another matter. Because they are based on chemical dyes instead of silver crystals, they are subject to chemical change (i.e. fading). Current films quote longevity of 50 to 100 years.
...laura
Re:Umm (Score:5, Interesting)
A minor fade can still be pretty bad. I found an envelope of 1980s-era colour prints as taken by my father - all seemingly of a number of people with cameras standing outside, near some flowerbeds and low fences.
On closer inspection, I noticed the very faint, faded image of the Taj Mahal [wikipedia.org] in the background, near-indistinguishable from the sky.
So, the photos are now useless, unless I scan them in and do some pretty heavy enhancement - but then what am I supposed to do with the results?
Re:Umm (Score:3, Interesting)
hell even the high end real exposed photos will not outlast the negatives.
I have a very expensive 8X10 print of a digital photo a friend shot back 4 years ago when he had access to an insanely expensive digital camera for that time. (your 3MP canon point and shoot can do the same thing it can now)
it is not exposed to sunlight directly and is behind UV protective glass in a frame and the yellow and cyan are already fadi
Re:Umm (Score:3, Funny)
"Archival" is probably code for "great for sticking in a lead box in a nitrogen bath in a sub basement for 1000 years without fading", but if you want the picture to be visible, well... You're not really archiving it now, are you?
Color - B/W (Score:5, Interesting)
Re:Umm (Score:3, Interesting)
It's not just a matter of the negatives, (color or b&w) so much as it's a matter of how they were developed. Masters like Ansel Adams & co. not only used better film, but they were also much more exacting in how the processed that film. Improperly stopped or fixed negatives (even when carefully stored) can deteriorate remarkably quickly....just ask a careless Photo 1 student. (not me, I was a careful Photo 1 student)
Re:Umm (Score:5, Insightful)
This is especially true if you print them out at home. Which makes me even madder that I fell for that "here's a cheap printer with a gazillion DPI" scam that Epson was running a few years back. Once I added the cost of photo paper and cartridges, it was more expensive than developing the pics.
Re:Umm (Score:3, Informative)
Printing on the printer costs me more, yes, almost twice as much as printing at the store. However, if you consider 250 pictures taken on vacation, I might want to print 10 of them as 8"x10", total cost about $20. Developing 250 pictures would cost at least $80, and I'd only get 4"x6"s, plus an additional $10 to get the 10 I want blown up to 8"x10" *after* I get the original prints back. The cost comes fro
Printing at the store (Score:3, Informative)
And they have 1/2 hour delivery, so they'd be done by the time you meander to the opposite end of the mall and back. For true convenience, there's internet sites that allow you to upload your pictures for printing, then they mail them to you.
Perpetual backups (Score:5, Insightful)
Re:Perpetual backups (Score:4, Insightful)
Re:Perpetual backups (Score:2)
Re:Perpetual backups (Score:3, Interesting)
And it depends on the value of the data. Music archives aren't really "worth" much, being mostly replaceable (even if you have to pay for it, the data is still available). The only music that has "value" in this context is something you've written yourself and not yet transcribed to paper, or computer-generated music that might be difficult or impossible to transcribe. But in most cases, you know what you did and can probabl
Re:Perpetual backups (Score:5, Insightful)
So what's going to happen a few years down the road:
Assuming you you kept your disks well, you'll end up with fine very fine disks of historical value which you won't be able to attach to any computer of the day. Can you read MFM and RLL disk today (assuming you still know what those were)? Do you still find a controller that fits into your computer? Is the operating system still able to read the file system (Anyone still remember Ontrac disk partition programs)? Do you still have programs able to open the files you saved?
If you decide to keep your data always online, sooner or later some accident will take care of them without you realising the fact.
If you trust storage companies, who do you trust to be still in business 30 or 40 years from now and still honoring your contract? If they go bankrupt you lost everything.
If you want to preserve some of your work to show your grand-children, you better take a backup medium that has been around for the last 30 years and still can be widely and read. This leaves us with
This might sound a little pessimistic, but my cupboard full of old floppy disks (8 and 5 1/4 inch, hard and soft sectored, CPM and other formats), 9 track tapes, optical disks and other mementos from work done in the past don't leave much faith in long term storage for digital data.
Re:Perpetual backups (Score:3, Insightful)
Assuming you you kept your disks well, you'll end up with fine very fine disks of historical value which you won't be able to attach to any computer of the day
But I don't give a crap about the disks, I want to preserve the DOCUMENTS. Having lost a lot of photos in a house fire I can tell you paper isn't near indestructible. And...
It's not like my documents aren't the very FIRST thing I copy over to every new computer I get. I still have source code I
Online Backups. (Score:3, Insightful)
If you figure that most people's data is under a gig. We're not talking about system images here, just the "my documents folder" and it's ilk. Less than a hundred megs/month with most people, including photographs, unless they really love their mpeg home movies.
With decent broadband and some system to do the backups during non-peak hours, you can easily do tens gigabytes a month. Will it cost? Yes, but it's like doing your own car repairs. Unless you have a garage equiped like the s
Re:Perpetual backups (Score:3, Insightful)
The only problem with that is finding the hieroglyph and cuneiform drivers for my brain.
Re:Perpetual backups (Score:3, Funny)
Anyone know if Kirstie Alley is still a Scientologist? Considering they believe body fat stores toxins and radiation and bad thetans (oh my), she must not be a very good Scientologist. Maybe she didn't get zapped enough. (Not meant as a slam on overweight people, but on L.Ron).
Permanent (Score:2, Funny)
Re:Permanent (Score:3, Funny)
WWLD? (Score:5, Funny)
--Linus Torvalds
Re:WWLD? (Score:3, Insightful)
Probably a good idea for a profitable service would be a gigantic digital safety deposit box.
Re:WWLD? (Score:5, Funny)
Re:WWLD? (Score:5, Funny)
Encrypt with GPG/PGP
Rename to "Olsen Twins Nude - XXX.zip"
Upload on Kazaa
What if all my files are pics of the Olsen Twins Nude? Do I have to upload them as "Linux Kernel 2.8 (preview)"?
Re:let 3rd parties store your data (Score:3, Insightful)
Keep em moving (Score:2)
I have files that are 15 years old purely because every time I move PC I copy all the data onto the new one.
Boingboing.net article contents (Score:5, Interesting)
Alien v Predator script saved by Internet pirates
Amazing anaecdote from Peter Briggs, the author of the screenplay for Alien Versus Predator.
Re:Boingboing.net article contents (Score:2, Funny)
Re:Boingboing.net article contents (Score:5, Funny)
Don't redistribute movie scripts! You might be partially responsible for 90 minutes of utter shite.
What I used to think (Score:2)
I think the biggest problem for me is getting around to converting them from the old format to the new.
If you think about the rate of growth in storage formats, you c
Re:What I used to think (Score:2)
Re:What I used to think (Score:2, Interesting)
Re:What I used to think (Score:3, Interesting)
The new Catweasel apparently also includes joystick/paddle ports and HardSID functionality. Yesss!
As far as beating bitrot by multiplying the data: You can also use software FEC encoding to add check blocks to the data, growing it by less than an integer multiple. Repairing the errored bits is automatic, whereas storing multiple
Wow... (Score:2)
50 years?? (Score:3, Interesting)
I decided to recover them and save the data on a CD, and I realized I didnt have a floppy drive installed on any of my machines! Somewhere in storage I had a USB floppy drive, but I cant get any software to read her files.
My solution: buy antiquated hardware.
No problem... (Score:5, Funny)
Let's just hope there isn't a fire or a flood.
Re:No problem... (Score:4, Funny)
Pick a scheme, any scheme... (Score:3, Informative)
This page [adams1.com] has a number of public domain schemes that might work. Some are fault more fault tolerant than others, but many of them should work.
Serious use of these schemes would require some kind of "Rosetta Stone" document or sculpture to make breaking the codes easy. If the archivist was to act carefully, I bet it would be possible for great-hoevermanygenerations-grandkid to break the bar code scheme, just by knowing a that the pattern is a rational symbology and by having enough repetition, of course. A
This is a non-issue (Score:2)
Re:Domesday Project? (Score:3, Insightful)
Formats (Score:5, Insightful)
But the main problem is not the "end of life" of media used for storage, is the format in which the information is. In 50 years, will be an application that opens/process that information? One of the advantage of having information in open formats is that in the worst case, you can have all the information to be able to process them. But if you stored your information using an applicaiton with its own propietary/closed format, and the company just decided to not support that format anymore, or just closed, you could have lost your information, even if the media where it is stored still retains it well.
Meaningless (Score:5, Funny)
Who gives a shit? I'm 39, and too mentally ill to attract a wife, so no kids. What am I going to leave behind? A collection of snotty and angry online postings? I just want to retire early and pursue my long denied hobby of global agitation.
And why doesn't the posting preview here work reliably with Firefox?
Thats it. (Score:5, Funny)
Perserving Electonic Data is oposit of Paper Data (Score:3, Insightful)
gee, nice ads on that link to Peter Briggs ... (Score:5, Insightful)
Re:gee, nice ads on that link to Peter Briggs ... (Score:3, Insightful)
Re:gee, nice ads on that link to Peter Briggs ... (Score:3, Funny)
Or did you mean there was a tiny, discrete ad for suicidegirls.com in the bottom right corner of the page? It's a good thing that slashdot would never do that [slashdot.org].
Re: (Score:2)
Holy Advertising Batman (Score:2)
I'll respond none the less... So far, I've managed to keep a good portion of my important information (writings, documents, pictures) that I want to keep around for a very long time.
While most of our pictures are slowly degrading over time. Data I've had for the last 10 years is still pretty much the same.
If its important, store it in more then one place...
Article kinda misses the real point.... (Score:2)
Kodak FUD?| (Score:5, Insightful)
Photos, slides and negatives don't last forever, just one look at the slides my Dad had in his house in Hawaii will illustrate that. But moving them to a new form of media is a lot more cumbersome moving 5 CD-Rs to a single DVD.
"Printing" is a bad way to save a picture, inkjet printouts degrade faster than true photos. You'd need to output to a real photo to get the same lifespan as a photo. Oh, and if you do, keep the digital copy, it's going to be better than a scan of the photo that's been sitting on the mantel.
Are there many consumers out there with more than 120GB of family digital photos? A spare hard drive is cheap these days as an additional place to store a copy.
Want to have your photos at home as well as somewhere safe in case of fire? It would be pricy to made dupes of all your slides or photos, but a second set of CDs pretty cheap.
There might be people who saved digital photos on floppys ( like those who got the cheesy Sony floppy cam ), but that media is not opsolete yet and for $20 you can have a USB floppy drive to let you move them to a CD.
Old media meant that the cost of the dupe was pretty much the same cost of the original. This doesn't lend itself to redundant copies at multiple locations for most people. Digital lends itself to duplication, just ask any movie pirate.
There are films from the 20's that are lost forever. Thanks to DVD pirates, we have enough redundant copies of Star Wars that it will never be gone.
Re: Kodak FUD?| (Score:4, Insightful)
NASA filled several volkswagens worth of magnetic tape with images and data from the 1976 Viking missions to Mars, but lost more than 20% of it due to tape decay. You can't just run out and buy a new hard drive to back that all up on, although they did try.
At one point during the effort to preserve old data, it was remarked that the tapes were degrading faster than they could be copied.
But what do you do with all that data once you have rescued it from disaster? You throw it on a shelf and wait until it is about to expire again while you move on to collect even more data. Eventually you will wind up as the modern day equivalent of medieval archivists who spent their entire lives trying make copies of books before they rotted away, only to start over again at the beginning as the first copies they made start to fade away.
What happens when you slip? What happens to a set of old CDs that get lost during a move, only to be found by your children when they are sorting through your attic after your death? Or what happens when the government decides that they can save some money by cutting back "wasteful" spending on data libraries?
It's all gone. Welcome to the Library of Alexandria. Please, no smoking in the stacks.
Re:My Tinfoil Hat Is On (Score:3, Interesting)
It was nice example of the market moving a big company away from a proprietary format. It probably won't happen with MS as more things out there keep adding support for WMA, but it at least shows it can happen.
Enough
The case for parity archiving? (Score:3, Interesting)
Damien
The ultimate storage meidum (Score:3, Funny)
Retrival may be a challenge.
Problem? What Problem? (Score:4, Insightful)
I make a backup of everything important once a year and take copy to my parents and the cottage. I take an incremental backup with me anytime I go visiting.
My kids will have bigger computers and any digital photos will just live on by being on their computers. And their grandkids computers and ...
Recently one of my aunts scanned in and touched all my grandmothers photo album. Now that album lives on CD and Hard-Drives of most of her 13 kids and 35 grandkids. Now nobody really cares who gets the original album.
Digital medium is SOLVING the problem of the loss of this type of heirloom data -- not introducing a problem.
I've got stuff almost 20 years old! (Score:3, Interesting)
Re:A few things (Score:5, Insightful)
But you have time to read a story on
Re:A few things (Score:2, Informative)
Pass "gaygaygay"
Hope that helps.
Re:A few things (Score:2)
Re:Quit whining (Score:4, Funny)
Complaining about posted links that require login isn't particularly new. Do you complain about comments like this EVERY time they're posted?
Re:Yeah, because that what we need. (Score:2)