New 25x Data Compression?

modapi writes "StorageMojo is reporting that a company at Storage Networking World in San Diego has made a startling claim of 25x data compression for digital data storage. A combination of de-duplication and calculating and storing only the changes between similar byte streams is apparently the key. Imagine storing a terabyte of data on a single disk, and it all runs on Linux." Obviously nothing concrete or released yet so take with the requisite grain of salt.

Limited application

[Locke2005]

Yes, it can compress data to 1/25th of original size... but it only works on slashdot articles, which are highly compressable due to the large amount of redundant data.

Re:Limited application

[Bull999999]

I, too, can compress data to 1/25th of original size... but it only works on slashdot articles, which are highly compressable due to the large amount of redundant data.

The proof...

[jforest1]

It's true! It compressed my 10GB collection of ASCII PR0N into 1 meg!

/dev/zero ?

[slimey_limey]

dd if=/dev/zero bs=1m count=1m | lzop - | gzip -f -| gzip -f - | gzip -f - | wc

gives about three kilobytes for a terabyte of data.

Incomplete Article Summary

[bigtallmofo]

The summary should have read...

StorageMojo is reporting that a company named Practical Nano Cold Fusion Duke Nukem Forever at Storage Networking World in San Diego has made a startling claim of 25x data compression for digital...

Re:Limited application

[LNO]

I, as well, can compress data to 1/25th of original size... but it only works on slashdot articles, which are highly compressable due to the large amount of redundant data.

Re:Breaking news!

[nizo]

Maybe it is lossy compression, which would be really nice when compressing executables and old spreadsheets.

Shame on you, ScuttleMonkey!

[RobertB-DC]

Posted by ScuttleMonkey on Wed Apr 05, '06 03:23 PM
from the make-sure-to-give-it-to-more-than-just-the- corporate-monkies dept.

You would think that an editor called Scuttle Monkey would know that the correct plural of "Monkey" is "Monkeys", not "Monkies".

"Monkies" would be the plural of "Monkie", which I guess is what you'd call a baby Monk Seal [wheelock.edu], or if you knew him really well, a resident of a Monastery [wikipedia.org]. "Hey, Monkie, nice robe!"

Of course, if you were talking to Michael Nesmith [wikipedia.org], the singular form would be "Monkee". But that's neither here nor there.

Re:What kind of data?

[swimboy]

It can compress anything! At the demo, I saw them compress 25 oz. of snake oil so that it all fit in a 1 oz. jar!

I've always imagined this conversation

[jfengel]

Developers: We've got some really good ideas for reducing backup space by using compression and incremental backups.

Marketing: How much in the best conceivable case?

Developers: Oh, I dunno, maybe 25x.

Marketing: 25x? Is that good?

Developers: Yeah, I suppose, but the cool stuff is...

Marketing: Wow! 25x! That's a really big number!

Developers: Actually, please don't quote me on that. They'll make fun of me on Slashdot if you do. Promise me.

Marketing: We promise.

Developers: Thanks. Now, let me show you where the good stuff is...

Marketing (on phone): Larry? It's me. How big can you print me up a poster that says "25x"?

This definitely works

[All Names Have Been]

I can tell you, this technology definitely works. I've seen them compress random data streams to 1/25th (even 1/30th!!) their size. This works *TODAY*. Coming out real soon now is the software that allows you to decompress your data. This is still in development.

Re:Limited application

[Alien Being]

Wow, *your* algorithm even compresses the moderation!

Re:Limited application

[sprag]

I, as well, welcome our 1/25th of original size overlords... but it only works on hot grits articles, which are highly compressable due to the large amount of petrified data.

Re:The proof...

[Dynedain]

The ASCII results:

*

Re:Limited application

[tshak]

I, wanting cheap karma, can compress data to 1/25th of original size... but it only works on slashdot articles, which are highly compressable due to the large amount of redundant data.

Re:Limited application

[Yvan256]

http://tinyurl.com/qhhqm [tinyurl.com]

Re:Where have we heard this one before?

[Anonymous Coward]

Please do not let this person ork any more cows. It's bad for him, and bad for the cow. Just say no to Cow Orking!

Re:What kind of data?

[networkBoy]

1.
I can compress anything you give me by a factor of at least 1 (inclusive of my own output).

"-1 pedantic", I know.
-nB

MOD PARENT DOWN

[gEvil (beta)]

Mod parent down! Nobody needs to see goatse again...

Re:What kind of data?

[morcheeba]

I've heard you have to do the decompression carefully, though -- If you do it too quickly, you just end up making a big mess.

Re:Limited application

[networkBoy]

dude, karma whoring funny comments is approaching the usefulness of this compression algo.
hate to break it to you this way :-)
-nB

Re:What kind of data?

[TheNetAvenger]

In other words, they're full of crap.

But the Slashdot Post says that is all runs on Linux. And knowing the infinite power of Linux, I believe them.

In addition to being the best OS in the world, Linux is also the most secure, does everything better than every other OS, and if given the right developers it is the ONLY os that could do something as impressive as compress data past the limits of possiblity.

I'm sure with the right developer, Linux could also be used to harness zero point energy, create wormholes for travel in your basement, and possibly cure most diseases... /wink

Re:Breaking news!

[x2A]

If only they had this a few years ago during the enron mess, they could have claimed "we didn't fiddle the accounts, we just saved it using lossy compression techniques".

Just like the "our intelligence wasn't wrong about Sadam having WMD's, the satalite images just come to us as lossy JPEGs"

(the point of this post lost due to compression)

Re:What kind of data?

[WhiteWolf666]

oOo. Sounds like you are going to find the data singularity.

A single byte that is all other data compressed together, and from which all knowledge flows! The universal black hole of data!

Don't tell me .... is this a new MS Vista technology?

Re:Limited application

[complete loony]

I, forgetting that funny doesn't give karma, can compress data to 1/25th of original size... but it only works on slashdot articles, which are highly compressable due to the large amount of redundant data.

Re:What kind of data?

[jedimark]

You mean /dev/random? :)

You geek!

[thepotoo]

Sheesh...when did you last get laid?