Big (and Small) Developments In Storage

louismg writes "On the same day that BlueArc released the Titan 2000 family, with performance more than three times higher than rivals EMC and NetApp, including a global namespace and scalability to 512 terabytes, EMC took a low-end approach by unveiling a line for the SMB market, dubbed Insignia. Red Herring claims that BlueArc's announcement changes the storage game, while The Register says that small means beautiful. What makes sense for today's IT infrastructures, with data growth showing no sign of slowing?"

Better organization!

[Myself]

More bulk storage only solves half the problem. As the volume of email and document archives grows, organizing and searching them becomes more interesting than just storing them.

Personally, I'm facing a minor storage crisis with a 15-gig music tree, a 20-gig photos tree, and dozens of gigs of other useful stuff that I only need on an occasional basis.

When I offload the camera, the files go to the laptop. To avoid duplicating files, the most recent few weeks worth of photos should always be on the laptop. But anything older should move off, to free up space on the drive. I'd like to keep as much of my music in both places as possible, but I need a way to replicate file moves, metadata changes, and deletions so the same bad rips and inaccurate tags don't persist after I fix them once.

There are a set of notes files, mileage logs, and other small files which I'd like to synchronize between the desktop and laptop. However, there's a possibility of both copies changing simultaneously (or at least between syncs), so I'd need a way to reconcile changes.

These little dilemmas have me wasting a lot of space until I resolve them. Stale data and useless files are all over the place, and I don't have the notes I need when I need them. It sounds like I might be able to do most of this with rsync, but the notes files, CVS maybe?

Has anyone tackled these issues before in a useful way, perhaps a sensible-storage-organization HOWTO? More space is not the issue right now, I need more sense.

Re:Better organization!

[jube_fl]

We are trying to over at http://www.storagerevolution.com/ [storagerevolution.com]. It is a new open source project created by a bunch of people with the same problems. Unlike the SNIA based aperi, it is not vendor controlled, it is totally controlled by users.

Data management is the largest problem we face, and the only tools to even start managing this problem are for the enterprise market. They are totally out of reach to the small to medium business, in both complexity and price.

We would appriciate everyones feedback as to what is important to them. This is the most valuable content anyone could deliver at this stage in the process.

Thanks