Is Storage Capacity Outstriping Backup Capability? 50
Kzip asks: "On my modest home LAN we have four computers with around 300Gb of storage. A lot of this is used, but not a lot of it is backed up (certainly not on a regular basis). When I started looking for a backup solution I found that most of the affordable tape backup was way to small (DAT 12/24 is just too small now a days) or too slow (Onstream do 50Gb but on IDE it's only ~1MB/s ... so 6 tapes over 80+ hours!) or just too expensive (HP Ultrium is great, but at £3000 for a drive and £120 per tape it's a little pricey). So I'd like to ask the /. community: Does anyone know of a fast and affordable backup system for home/small office use." After a quick scan of Pricewatch and other sites, it seems that backup solutions >99G are expensive (all the ones I could find were more than $1000US). How long will it be before these and
terabyte-backup solutions become affordable for SOHO ? use?
Reduce risk with a backup harddrive (Score:1)
The other alternative is to only have data that is crap, like this post, which isn't worth backing up.
Re:Reduce risk with a backup harddrive (Score:5, Informative)
By identifying yourself as a SOHO user, you have implicitly set your level of tolerance for backup failure. You just want safe recovery of your data in the event of a malfunctioning disk; you don't need to prevent against natural disasters, etc.
Dedicated backup hardware (i.e. tape drive arrays) costs too much - you already know that. Because you don't *require* all the securities offered by tape backup (i.e. the ability to dump data, store media in an offsite location, etc), the best backup solution would simply be a machine located on your existing network (yes, 100Mbit ethernet is fine) with enough hard disk space to hold your data. Remember, its your backup process that needs to be fault tolerant, not the backup machine itself.
So, if you have an extra Pentium or Pentium II lying around, equip it with an IDE Raid card and enough IDE disks to hold your data. I would suggest RAID 0 - well supported, fast, and inexpensive. Install your OS of choice (this is slashdot, so i guess its linux for you. My personal inclination would be a BSD, and in this case I would choose FreeBSD, simply because my experience with OpenBSD and IDE disks hasn't been great). On Day 0, completely synchronize the backup machine (i.e. cp -R
Re:Reduce risk with a backup harddrive (Score:1)
P1. You are a SOHO user.
C1. You just want safe recovery of your data in the event of a malfunctioning disk.
C2. You don't need to prevent against natural disasters, etc.
C1 and C2 don't follow from P1. Just because Kzip is a SOHO user doesn't mean his backup needs are limited to protection against disk failures.
I think hard disk storage is outstripping backup capability. For under USD1000 I can easily add 300GB of raid 5 storage. Backing that up to tape in a reasonably timely fashion (say 6-8hrs unattended overnight) is prohibitively expensive. Let alone the added media costs associated with the not unreasonable requirement to be able to do backups on a regular schedule (such that I can rotate a backup off site at least once a week and archive monthlies).
In a SOHO environment, I can easily imagine the desire to maintain off site backups--your apartment could burn down, someone could break in and guess what? rip off your nice ide raid file server and other computers, some freak lightning strike might blow right past your surge supressors and fry all the computers on your SOHO lan, etc. etc.
Tape backups also help protect against accidental/malicious data destruction. For argument's sake, say the poster's 300GB of storage was on a Mac. Partitioned. And then he ran the new iTunes =) kaput, raid or no. And no backup to go to. Anyway, you could come up with similar/better examples with a little thought--virus, cracker, well-placed bottle of beer....
Offsite backup exchange ring? (Score:2, Interesting)
I got to thinking once again about how fragile my current backup scheme is.
I have three machines -- one laptop for MS Office, games, and browsing the rare website I can't use with Opera on Linux, one ancient Linux box doing NAT and email, and one modern Linux workstation that I use for my daily work. My backup sets from these machines total about 15 gigabytes compressed. About five gigabytes are irreplaceable data, and replacing the rest would be several days' work.
I take backups every night. The two linux boxes cpio, gzip, and cp new and changed files off their relevant filesystems to a separate backup drive on the workstation, while the built-in backup applet on the Win2k box takes a full backup onto the same drive via SMB. An additional cron job on the workstation renames the windows backup file each night so the new one doesn't.
This scheme protects me from any single drive failure, as well as accidental deletion of any file or directory except the root directory on the Linux workstation. Which means that if, for example, if I were to install a poorly constructed RPM that did rm -rf $DIRECTORY/ in a script while $DIRECTORY was unset, I would lose all my data. While I do have the most critical stuff on CD's, those are usually weeks out of date. And even those would be susceptible to a housefire or a burglary (well, I don't know if burglars typically bother to steal stacks of CD-R's -- anyone with experience in this regard?).
In any case, offsite backups would be the way to go. CDR's are a pain to automate when your backup set is large, and a pain to drag off site, and they can get quite expensive in the long run, so I've looked into various online backup services. Unfortunately, at the volumes I'm looking at, the commercial services will typically charge you several times the one-time cost of the hard drive space each month, which seems somewhat excessive.
In fact, if I could just find two or three random guys with DSL or cable, each with a server that's always on and has 20GB to spare, I'd be quite happy to give them each 20 gigs on my box in return. A 15-minute search on Google didn't turn up any backup exchange rings on the web -- is someone doing this kind of thing, either privately with friends or through a more open group? What kind of software do you use? While I would be perfectly happy with cpio | gzip | gpg | ssh cat to stash my own stuff, I would be hesitant to give random strangers full shell accounts on my box. And I would prefer not to let them turn my workstation into a warez server, either, although I suppose IP address restrictions and monitoring would pretty much take care of that. Something that runs on windows would also increase the user base nicely.
Has anyone been thinking carefully about a peer-to-peer online backup system, or should I?
Re:Offsite backup exchange ring? (Score:1)
Also, when I got my modem installed this summer, the tech gave me a lecture about bandwith, and warned me that people often get ports blocked for bandwidth abuse. I could see such a bandwidth deal resulting in nasty things happening to your connection, like getting ssh blocked.
Re:Reduce risk with a backup harddrive (Score:2)
ATA raid controller: $330
4*100 Gig drives: $800
RAID 5 backup: cheaper than tape
Re:Reduce risk with a backup harddrive (Score:1)
why in the world would you spend $330 on an ata raid controller when you can use software raid for free? it's probably faster, too.
btw, some of those ata raid controllers are nothing more than normal ata controllers, they just use software raid drivers (patched into the x86 bios calls, i believe).
Re:Reduce risk with a backup harddrive (Score:2)
Re:Reduce risk with a backup harddrive (Score:1)
do you think the cpu in that card is fastest than the one on your mobo? some raid controllers have battery backed ram, which is a plus (though a large ups in front of the whole system is about as good).
For $330 you get native raid 5 support and the peace of mind knowing when your OS fails, the raid array maintains it's integrity.
i didn't mean to imply that true hardware raid controllers aren't worth it. they are, in some cases. it's the fake ata raid software driver based cards that aren't worth it.
Tell me what software rendering looks like in comparison to a GeForce 3 Ti. I'll take my RAID in hardware, Thank you.
that is not an argument for hardware raid. hardware raid cards use slow cpus. graphics cards use custom asics that are much faster (as you state) at certain tasks than a general cpu.
Re:Reduce risk with a backup harddrive (Score:1)
Okay I'll ask, does anyone have examples of 'true' hardware raids? What about the Promise FastTrak cards? I would like to buy an ide raid controller card to do raid 0 and was considering one.
Re:Reduce risk with a backup harddrive (Score:1)
Each supports 4 drives
I might make the observation.... (Score:2)
big backup costs big $$$ (Score:4, Interesting)
In a word, yes.... (Score:5, Informative)
I still think it's crazy to pay $140/tape for SDLT at the office....
Re:In a word, yes.... (Score:1)
works.
The major cost was in the software, not the hardware. We looked at the free software option Amanda [amanda.org] but it was too linux/unix specific.
We needed support for Win2k, NT, Solaris, Linux, Oracle DBs and more. So we had to get the serious cost software.
And we only specificed a 4 TB backup capacity (though with some room to grow). That's not a particually large lot of data these days.
The suggestion of just mirroring stuff to another disk is a good one, but doesn't really provide enterprise level backup. Unless your mirrored disk is off-site. Then you just move the cost from backups to networking
RAID maybe - but dont backup HD to HD... (Score:3, Interesting)
I think if you want to do backups to HD you need three of them!
Re:RAID maybe - but dont backup HD to HD... (Score:1)
Just think what happens when the source disk fails in the middle of the nightly backup. You have a failed source disk. And a half-baked backup disk.
ever hear of incremental backups?
Re:RAID maybe - but dont backup HD to HD... (Score:1)
And the answer is... (Score:2, Funny)
Yes.
You're welcome. Any time. Glad I could help.
Re:And the answer is... (Score:1)
And what about non "enterprise class" business? (Score:2, Insightful)
If I was a more enterprising geek....
Check into Intervault (Score:2, Interesting)
Big Cheap Tape Drives (Score:4, Informative)
I have one of those and it's plain cool.
Re:Please include link in message (Score:1)
Here's the link: http://www.ecrix.com/ [ecrix.com]
BTW, speaking of .sigs, Slashdot's added a new feature that makes them less annoying: You can now
enable "sig dashes" in your user profile [slashdot.org] so that
all sigs get separated from the body text with a
--<BR> or the like. Makes many sigs
much less annoying. (Since you're an A.C.,
I don't suspect it benefits you much, but I
thought I'd mention it nonetheless.)
--JoeUser a smart backup strategy... (Score:4, Insightful)
You just don't have to backup
can just reinstall them from the CDs they came on.
-> Just save a list of your installed RPMs
(redhat has a script for that purpose, I'm sure
debian, slackware,
And your 50GM collection of MP3s doesn't change either. So just save them to CDr, which you can
stick in your cheap DVD-player for easy listening
on your home-stereo.
So just make some permanent backup of things
that will not change and incrementally backup
only things that are changing.
I doubt that your current programming project,
your mailfolder and other things that change
often are more than you can fit on a DDS3 DAT Tape...
And if your computer breaks, you just reinstall
your OS from your saved config (insert the CD,
wait 15 Minutes, you can make yourself a pot
of cofee in the meantime), when it's done
you add your CDs (which of course have the
proper location the data on them wents stored!)
while the DAT fills your
backup and your'e set.
Re:User a smart backup strategy... (Score:2)
1. Set up your partitions in a way that facilitates backups. For instance, make
2. Have an actual backup schedule. Even if it takes 80 hours to back up your entire system you can still do 1/20th per night in four hours and have four hours left to do incrementals everywhere else.
Oh, and cheked out, but unchanged CVS files should not be picked up by an incremental. It is based on mtime not atime.
Finally, "prioritizing" files for backup is antithetical to the purpose of most backups. There may be some good uses for this, but it is going to be a small niche
-Peter
Most people do not have 300GB of 'data' (Score:1)
But to answer the question: Yes - storage is overtaking backup capacity. A new approach is required. Some sort of writable DVD is probably the solution.
Re:Most people do not have 300GB of 'data' (Score:1)
This I can believe.
None of which is critical, and therefore does not need to be backed up.
This is bunk. If I had 390GB of images, movies, and MP3s that I had taken the time to seek out and download, they're critical. Bandwidth costs money, and the time to find the images, movies, and MP3s all over again costs money. Legal issues aside, ya damn well better believe that my time and money is critical, and therefore -any- data.
Re:Most people do not have 300GB of 'data' (Score:1)
Pricwatch.COM (Score:2)
Use other computers as backup (Score:1)
Since you've several computer connected to the network, reserve some place on the hard-drive to backup information from the other computers (a folder or a partition, whatever). Decide what you need to backup (ini files,...) and write a script to do it for you (in bash under Linux, or a .bat file under windoze). Write the script so that it erases the previous backup only after the current one is finished. Just start this script when you feel a need to back up. ;) (almost did once)
I've done this for my email archive. I wouldn't want to lose it in a crash.
This solution is the best for you in your current network configuration. If one computer fails, you'll still have your data available on one of the others.
For those with only one computer, if you have two HDs (not 2 partitions on the same drive), the solution is to use the second drive to backup (data from) the first, and the first drive to backup the second (you usually don't install programs only on the first one).
My last tip: use zip or rar to reduce the size of the backup (and the number of files). It's a little longer to do, but you'll backup more in less space.
traditional backups to the cheapest reliable media (Score:3, Interesting)
All the Unix backup tools can backup to disk as easily as to tape. Carriers to make ATA/100 disks removable cost about $10 each. ATA/100 disks are cheap per megabyte.
There are techniques to make the disks hot swappable, or use a dedicated backup machine that can be easily powered down to swap disks.
Most importantly! It's a restore system, not a backup system.
Nobody cares how great your backups are, if you can't do a restore when you need it.
My solution... (Score:1)
Build PC with 3+ partitions (c:=system d:=apps e:=data f:=MP3/misc g:=games)
I make sure that c: is a relatively small partition (2GB or less). I run Norton Ghost to make an image of the C drive. Typically, the C image will fit onto a CD. I do the same with the D drive, assuming it's not too big. E I just burn straight onto a CD, F I mostly have on CD already, so I just burn the additions every 600MB or so. G I don't worry about.
In the event of a disaster, I just use the C and D images to restore those two partitions, and start copying CDs to restore the others.
Re:My solution... (Score:1)
But what about your Diablo save game files. I'd hate to lose mine...
Not enough data. (Score:2)
You don't have enough data to ask that question. Tape backup is still the cheap way to store a lot of data in a small amount of space. however if your tapes don't take up a football field then the need for small cheap storage doesn't really hit you.
Backups have several advantages byond the above: timed snap shots. You generally keep several copies of your data from different times, realize you made a mistake several days ago, you can go back to before that mistake.
Tapes are easy to move off site. Critical data must be moved offsite. Preferably several copies.
Backups for purposes of dealing with yesterday's mistakes are better delt with via good version controll. Get and use version controll on all your documents.
Now you only need to protect against hard drive crashes, and nateral disasters. I recomend a good insurance policy. Don't protect jut the equipment, protect the income lost tryign to re-create your data. A $100,000 disaster insurance policy isn't that expensive (but you should seriously consider more!), and you need it anyway, along with thief protection.
Hard drive crashes in small systems are best protected againsts by mirroring. Copy all your data to anouther harddrive, they are cheap enough that this solves most of the hardware failure problems. I recomend a small computer locked in a basement closet, so that theives don't get it.
Once a month or so decide what is really critical and copy that to CDROMs (DVDrom?), which you store at your parents. You can get your MP3s again. You can take anouther picture of the leaning tower, so don't save it. (unless your kid is in the picture, since you can't get anouther picture of your kid at that age) Buisness data doesn't all have to be kept. Just enough that you can reconstruct your buisness. Your suppliers will be happy to send you a new price list. Linux (or whoever is now maintainign the stable kernel) will be happy to give you a new kernel.
It takes a lot of data to make a $10,000 tape drive doing 50GB at $20/tape pay for itself, and you really should seriously consider a robotic library for even more $$$. You can do the math. If you are in that crowd, StorkageTek (the company I work for) will be happy to sell you such a system. We admit freely that when you have less then 30 terabytes of data tape backup often isn't the solution.
300 Gig!? (Score:4, Informative)
Seriously - I couldn't care less what you had, but you need to ask some serious questions here. You talk about four computers with 300 gig of storage, so that is around an 80 gig drive per machine. What you first need to do is consolidate and eliminate duplicate material - ie, build a fileserver, and eliminate redundant data.
How many of those MP3s are kept local on each machine, as copies, etc - when there should only be one copy? Same with those mpegs and jpegs, and any other kind of data.
When and where possible, drop as much of that data to CDs, and remove it off the hard drives - in fact, if I was in your position, I would build a machine with four of those 80 gig drives, then drop small 8 gig drives in each local machine. Partition that 8 gig drive into a 2 gig system partition a four gig application partition, and a 2 gig data partition. Give each user space as well on the fileserver. Put all the MP3s on the fileserver, and hook everyone and the fileserver up through a 100Mb switch. Also, each user can backup their data on their data partition to whatever medium suits them (to the fileserver, to a floppy, to a CD - whatever suits the amount of data they have), and forget the rest (in the event of a real problem, it can be reinstalled from the original disks, or from a backup on the fileserver).
You may also want to partition the fileserver, depending on the type of data being stored (or simply keep certain data on certain drives). Then, decide what is important, and what isn't (is an MP3 important - or is that 300 page dissertation important), and backup the important stuff to CD. Perhaps build a second machine to act as a "mirror" of some sort.
None of these suggestions should substitute for a real backup solution - so you can only do what you can with the money and stuff you have. But there is a way to keep most of what you have safe enough...
Re:300 Gig!? (Score:1)
No need to waste time and space including MP3s in your regular backup schedule.
Removable Hard Drives? (Score:1)
They are in Aluminum Removable Caddies. I turn the key, (which powers the drive down) turn the handle, and remove. I pop in a different drive, turn the key, and w.2000 thinks a few seconds, and accepts the new drive.
This is a good removable backup solution. (As long as you don't drop the drive on your way to the fire safe)
Has anyone seen or heard of a removable IDE system? That would be much more affordable...
Even at a few hundred $ per drive, it beats tape on all counts.
-Speed
-Price per MB
-User Friendly (No multiple tape backups, one drive does it all...)