Become a fan of Slashdot on Facebook


Forgot your password?

Submission Ask Slashdot: Is the gap between data access speeds widening or narrowing?

DidgetMaster writes: Everyone knows that CPU registers are much faster than level1, level2, and level3 caches. Likewise, those caches are much faster than RAM; and RAM in turn is much faster than disk (even SSD). But the past 30 years have seen tremendous improvements in data access speeds at all these levels. RAM today is much, much faster than RAM 10, 20, or 30 years ago. Disk accesses are also tremendously faster than previously as steady improvements in hard drive technology and the even more impressive gains in flash memory have occurred. Is the "gap" between the fastest RAM and the fastest disks bigger or smaller now than the gap was 10 or 20 years ago? Are the gaps between all the various levels getting bigger or smaller? Anyone know of a definitive source that tracks these gaps over time?

Comment Changing Requirements?? (Score 2) 203

Did the project fail because of incompetence on HP's part...or did the customer (the government in this case) keep changing the scope and requirements so often that it was impossible to actually do what they wanted? I know nothing about the details of this particular case, but either condition (or both) would not surprise me as the cause for the failure.

Comment Re:Color me impressed (for real)... apk (Score 1) 114

I was one of 4 programmers that wrote the first 3 or 4 versions of PartitionMagic. I was the only programmer on the Drive Image team for its first 2 versions. After it started bringing in a few million $ in sales, the company (PowerQuest) decided to add some other programmers to the team to help me :)

Comment Working on the next big thing right now by myself (Score 1) 114

Well, not quite...I do have one other guy helping with a GUI admin tool that calls down into my system. But all the guts of my new, general-purpose data management system were written exclusively by me. I have a huge list of features yet to implement, so I could use a lot of help, but until someone steps up and wants to dive in with me, I am on my own. It is an incredibly ambitious product (think file system, relational database, key-value store, graph database, and distributed data management system all rolled into one big data object store that uses multiple data models) so there is nothing trivial about it. I have been in the data management business for 30 years (wrote file system drivers, custom file systems, PartitionMagic and Drive Image, cloud backup, etc.) and I have never seen anything that comes close to my system in terms of speed and flexibility. At this rate, it might still be more than a year before it is ready for use in a production environment but I still code on it every night and weekend. A video of my demo can be found at

Comment Re:When are we getting a taggable filesystem? (Score 1) 132

Although the UI in the demo is running on Windows, it is built with Qt so it is easily ported to other platforms. I plan on getting it running on Linux and OSX as well. I know that need to get some better video of the demo as well. I have just been concentrating on getting features working a lot more than on trying to promote it.

Comment Re:How about this (Score 1) 66

That might be how others have implemented an object store, but not how I am doing it. There are not two separate systems to manage in my case. I don't use a file system to store the unstructured data and store all the metadata in a database and thus have two different systems to try and keep in sync. Instead, I built a new system from the ground up (actually from the block pool or disk partition up). It stores structured data and unstructured data natively in a very unique fashion.

Comment Re:How about this (Score 1) 66

Working on it. And unlike the "storagedude", I do have the expertise to implement it. I have it about half implemented so far. I have created an object store where every object can have lots of attributes or tags attached to them. Unlike extended attributes, you can actually find things based on them quickly. For example, I can create a container and put 100 million of my data objects in it (photos, mp3s, software, documents, etc.) and find anything and/or everything in just a few seconds. If I had 10 million photos and they all had tags attached, I could find the 50,000 photos of my family vacation in Hawaii in 2010 in just a couple of seconds. Check out the video of my demo at

Comment Re:Funding Needed (Score 1) 132

On the contrary....I have had several people tell me it is something they need. They wish it was finished right now. But for whatever reason, they won't put any resources toward getting it finished. I have put a lot of personal time and money into the project (a couple hundred thousand $) but it is still only about half finished. It is a very ambitious project. I finally had to go back to work to feed my family, but I still work on it in my spare time. I am determined to finish it, but it will be years later than it could have been with some funding.

Comment Re:Funding Needed (Score 1) 132

Good luck with that. Nobody seems to want to pay for system level software anymore. They might shell out a few bucks for a game that they will grow tired of after a few weeks, but they expect their OS, tools, and other platform software to be free (as in beer). You might build a system, library, or algorithm that collectively saves the world economy a $ billion dollars per year in saved time, electricity costs, and/or hardware upgrades; but don't expect to get paid anything for doing it. Sad as that might be, it seems to be the reality we currently live in. For the past few years I have been building a new data management system that blows the doors off a bunch of file systems and databases in terms of performance, but I can't get anyone to fund any of it. They all have the same reaction to the demo of my partially working system..."That is amazing. Let us know when the development is complete and tested in an enterprise environment and we will use if we can have it for free" (as if I had a bunch of magic slave elves coding for me all night in my basement in order to accomplish those things without any resources).

Comment Open source is not always the best option (Score 5, Interesting) 316

Some people seem to be under the impression that free software is always a better choice than proprietary software. Some of the stuff released as open source software is garbage and there is often little or no incentive for those who wrote it to fix it. There is also a lot of good stuff out there with wide community support as well. I have used a lot of open source AND proprietary software and there is a lot of good and bad stuff in both camps. It is amazing to me how many people will spend many hours and extra training costs in order to get something working just so they don't have to spend $20 for a license to something else that works a lot better. If I find some really good software and the guys who built it want $50 from me for their efforts, I am happy to pay it. My time is worth something.

Nonsense. Space is blue and birds fly through it. -- Heisenberg