Journal chill's Journal: Personal Data Archives 3
I've written here before about how I rip and store every bit of physical media that I purchase. Movies, television shows (on disc), music, and pictures, are all copied and stored on a little server I have in my home. Everything is available on my home network, via Samba shares, with televisions being connected to CuBox devices running LibreELEC for an interface.
All in all I have about 2.5 Tb of data, with over 600 movies and TV show episodes -- all legally purchased -- in SD or HD resolution. It was stored on a single drive in a small server I had in a utility closet. Backups were made to a USB drive, stored elsewhere. The ultimate backups are the original media.
Over the Thanksgiving holidays, it all imploded. I was sloppy and lazy and got burned. The server was not on a UPS and a nasty power surge took out both the motherboard and the hard drive. I was unable to get the drive to spin up at all. Any machine it was attached to wouldn't even enter BIOS/UEFI until after the drive timed out, so even SpinRite was useless.
My USB backup drive, of course, failed. I don't know why, and I hadn't used it in several months, but it had a ton of errors. I was able to recover about 5% of my data from that. The rest, I've been recovering from original media. I expect to finish some time in late January, because that is a slow, tedious process, and because I'm obsessive about getting the encoding right and the tags correct.
As before, I'm using MakeMKV for DVD and Blu-Ray rips, with HandBrake for encoding. Under Linux, these do an excellent job, but I can't get the tags to auto-populate like it is possible in Windows, so it can be slow.
To ensure this doesn't happen again, I'm doing the following:
1. Adding a UPS to the main server. And, instead of a normal Linux box, I'm expecting one of these to be delivered early in January. I'm planning on configuring it as RAID-5 with some 4 TB WD Red NAS drives, for a total of 12 TB of usable space.
The Internet is full of stories on how RAID-5 is dead and a bad choice for modern, large (multi-TB) NAS, but my use case is different. In my case, it is much closer to WORM than traditional NAS. I'll be writing to the system maybe once a month, as I add new content, and reading frequently by not continuously. It should serve my purpose without having to dedicate 2 drives to redundancy.
2. I'm also adding a second layer of backup in the way of BDXL 100 Gb M-Disc archives.
Surprisingly, Ubuntu doesn't properly support recording Blu-Ray discs out-of-the-box. It seems that some time ago there was a major pissing contest around the license between the original author of cdrtools and some distro maintainers. The end result was forking some tools under the CDRKit package.
TLDR -- My first attempt resulted in a $17 coaster, as 100 GB BDXL M-Discs aren't cheap.
For Ubuntu, follow the instructions here to add a PPA and then install both K3B (main repository) and the needed PPA tools, cdrecord and mkisofs.
After doing that, I can now successfully record 100 GB backup discs using K3B. It takes about an hour-and-a-half for them to burn, using an LG WH14NS40 drive. And while a 25-pack of these beauties will set me back about $375, it beats the pants off of 3 months worth of re-ripping and encoding.
For anything else, I'm comfortable just encrypting and uploading to Google Drive. But 2.5 TB of media is just too much, and the actual upload and download times are too damn long.
The original M-DISC company went bankrupt, and assets were acquired by the debtors, as documented in the Wikipedia link above. While the *idea* of long term, stable, archival storage in a DVD/BR media is fantastic, I think they bumped into a the reality of there's only so much data that I need to store long term. Once I bought a set of 10 DVD blanks, back when it came out, I only really used 1. Once you get rid of things like videos, music, and pictures, the rest of the data I generate and care about is significantly smaller than 4.7 Gb. I still have 7 of the original discs I purchased back in 2011, when I got my first M-DISC capable DVD-R. (Two were experimental coasters.)
Aye, there's the rub ... (Score:2)
I was moderately interested at 100Gb/per. That's just killed it dead. But you've made me (appropriately) paranoid about my backup on USB-disc. Something to attend to urgently when I get home from the holiday.
Re: (Score:2)
Verbatim and a couple of others have license for the M-DISCs. LG and others have license to make the drives. I've had no issues over the years finding both burners, internal and external, and discs.
The magic is the discs conform to standard DVD/Blu-Ray specs, they just use a different, rock-like substrate instead of plastic. That means they can be *read* in any standard DVD or Blu-Ray player. It is just *writing* them that requires a drive with a more powerful laser.
But, yes. Backup, backup, backup. 3-2-1.
Re: (Score:1)
All my USB disk problems were traced to the crappy cable and connector. I would take the disk out of the box and install it internally. I've been lucky so far. Now I just do a poor man's RAID 1, buying two of everything. Internal drives are cheaper, so I found this really cool docking thing [startech.com]. Hope it works...