jornal checksumming only prevents errors in the journal, not once the data has been written to the main storage area. This was done primarily to ensure the atomic nature of the journal is not violated by a partial write.
Our BTRFS evaluation resulted in rejecting it for some very serious problems ( what they claim are snapshots are actually clones, panic in low memory situations, no fsck, horrible support tools, developers who are hostile to criticism, pre-release software,
Unless every read does a checksum ( they don't or it would kill performance ) then there is still the possibility of a silent read corruption. At 70TB it would be rare, but not as rare as many would think and would depend on the sector size and checksum on the individual drives.
A much better test of linux "big data"
1) write garbage to X blocks
2) run fsck if no errors found, repeat step 1
How long would it take before either of these filesystems noticed a problem and how many corrupt files do you have? With a real filesystem you should be able to identify and/or correct the data before it takes out any real data.
After evaluating our options in the 50-200TB range with room for further growth we ended up moving away from linux and to an object based storage platform with a pooled, snapshotted, and checksummed design. One of the major reasons for this was the URE problem, we would virtually be guaranteeing silent data corruption at that size with a filesystem that did not have internal checksums. The closest thing in the OS world would be ZFS whose openness is in serious doubt. It is scary how much trust the community places on spinning rust.
The tests are also useless since the "speed" will be linerally controlled by the IOPS of the array. Sure would be nice to be able to throw 10x15k spindles at 3.5TB ( 230 disks for the 72TB test ) that's one way to improve random IO performance, but how many can afford such luxury on a big data store that could reach into the 100's of TB?
Personally I think people are just not ready to accept the fact that high overhead, low margin, stores like bookstores are a dying breed and not likely to survive long term. They are on the short end of the "innovation" stick and it's only a matter of time before they become extensions of their profitable online counterparts or go out of business. They require too much of a footprint for the small profit margins. Between amazon and ebooks, the traditional bookstore is the buggy-whip of the the 21st century. With the death of the traditional bookstore will also usher in the death of the traditional publishing conglomerate and I will not shed any tears for those rentiers.
Personally I think there is still room for retail innovation, but people have to be willing to think outside the box to make something great. Think the best of the coffee shop, tiny lending library, amazon kiosk, and/or on demand paperback printing. Something that adds a tiny overhead to the profitable coffe shop model while adding the best points of the bookstore. Heck, work with local government to partner with the library system so that you don't even need to own the lending books.
You should ask Macys how that's working out with the Martha Stewart collection. Arbitrage is a bitch.
Don't bother adding *real* value, just make it harder for the consumer in the long run. This will end well.
To me the biggest security win with VM's is the ability to properly size a system for what it is actually doing. No more adding "just one more" service to a box because it's got more horsepower than it needs. By doing more logical partitioning of the software you limit the commingling of data, administration, and crash risk between different services.
Oh, and to output a pipe to multiple processes you just need to use tee.
tee >(process1) >(process2) >(process3) | process4
After which you could use something to join the files back together. I've used such a syntax to pipe the output of an application over ssh while also taking a sha1sum without the need to save it locally.
Psst,
" | sort | uniq -c "
Will sort and then count repetitive lines and output count, line. You can pipe the result back through sort -n if you want a frequency sort or sort -k 2 for item sorting.
Bad summary. 1B gals/year is quotes in the article.
Maybe thats just me.
The N900 is a limited success *despite* Nokia's best effort, just like the "NIT"'s before it. I mean come on, the firmware has bugs that are 2 generations old and still stewing. Development for the device is a joke since it was ( last I checked ) still basically tied to running debian or ubuntu on your development system unless you wanted to use Python.
I have developed for the N810 and was sponsored by Nokia to the first Maemo conference. What I saw and heard lead me to the conclusion that it was a dead-end without partners. MeeGo *might* change my view if they can finally bring a mainstream success to the table without the continued alienation of their development community.
The N8 that will ship soon is slightly different hardware wise, and vastly improved software wise.
Color me skeptical; what are the chances this is not another overpriced "flagship" POS from Nokia?
I think what GP was trying to say is what if the malware has altered the the reported amount of memory in the system and is now hiding in the chunk that neither the bios nor the OS believes exists?
How about hiding in any number of IO mapped chunks of memory like the BIOS or Video Ram?
I've noticed several design suggestions in your code.
