Become a fan of Slashdot on Facebook


Forgot your password?
Slashdot Deals: Deal of the Day - Pay What You Want for the Learn to Code Bundle, includes AngularJS, Python, HTML5, Ruby, and more. ×

Comment Working on similar feature (Score 1) 83

I am building an object store where some of my data objects can each be a key-value store that is used as a column in a relational table. Some queries against a table require a full scan (e.g. SELECT * from my_table where address like '%Main Street%';). If I had a table with a billion rows, it can take awhile to scan the whole address column looking for matches (I dedup the values in each column, but there can still be 100 million unique address values in such a table.) The solution is to break each column into multiple segments and let separate threads scan each segment looking for matches. The scan can occur in parallel on multi-core machines and complete in a much quicker manner than forcing a single thread to scan the whole thing. It sounds very similar to what they are trying to do with PostgreSQL (except that database is row based where the whole row is stored together, instead of a columnar database like mine). Here are two short demo videos of the system in action.

Comment On the Internet...nobody knows your a dog... (Score 1) 696

...or a woman...or a kid...or a grandma...or black...or gay...or whatever. Personally, I have never looked at code or used a library and thought...this looks like a woman wrote it. Code works or it doesn't. Who cares who wrote it? Some women may face discrimination in the workplace. Some may be hired and/or promoted simply because of their gender. I once had a woman on my team who wanted the company to lay her off for the severance package. She did just about everything she could do...come in late...leave games...etc.. The company instead let other men go at layoff time. They wanted her in their stats.

Submission + - Ask Slashdot: Is the gap between data access speeds widening or narrowing?

DidgetMaster writes: Everyone knows that CPU registers are much faster than level1, level2, and level3 caches. Likewise, those caches are much faster than RAM; and RAM in turn is much faster than disk (even SSD). But the past 30 years have seen tremendous improvements in data access speeds at all these levels. RAM today is much, much faster than RAM 10, 20, or 30 years ago. Disk accesses are also tremendously faster than previously as steady improvements in hard drive technology and the even more impressive gains in flash memory have occurred. Is the "gap" between the fastest RAM and the fastest disks bigger or smaller now than the gap was 10 or 20 years ago? Are the gaps between all the various levels getting bigger or smaller? Anyone know of a definitive source that tracks these gaps over time?

Comment Changing Requirements?? (Score 2) 203

Did the project fail because of incompetence on HP's part...or did the customer (the government in this case) keep changing the scope and requirements so often that it was impossible to actually do what they wanted? I know nothing about the details of this particular case, but either condition (or both) would not surprise me as the cause for the failure.

Comment Working on the next big thing right now by myself (Score 1) 114

Well, not quite...I do have one other guy helping with a GUI admin tool that calls down into my system. But all the guts of my new, general-purpose data management system were written exclusively by me. I have a huge list of features yet to implement, so I could use a lot of help, but until someone steps up and wants to dive in with me, I am on my own. It is an incredibly ambitious product (think file system, relational database, key-value store, graph database, and distributed data management system all rolled into one big data object store that uses multiple data models) so there is nothing trivial about it. I have been in the data management business for 30 years (wrote file system drivers, custom file systems, PartitionMagic and Drive Image, cloud backup, etc.) and I have never seen anything that comes close to my system in terms of speed and flexibility. At this rate, it might still be more than a year before it is ready for use in a production environment but I still code on it every night and weekend. A video of my demo can be found at

Comment Re:When are we getting a taggable filesystem? (Score 1) 132

Although the UI in the demo is running on Windows, it is built with Qt so it is easily ported to other platforms. I plan on getting it running on Linux and OSX as well. I know that need to get some better video of the demo as well. I have just been concentrating on getting features working a lot more than on trying to promote it.

Comment Re:How about this (Score 1) 66

That might be how others have implemented an object store, but not how I am doing it. There are not two separate systems to manage in my case. I don't use a file system to store the unstructured data and store all the metadata in a database and thus have two different systems to try and keep in sync. Instead, I built a new system from the ground up (actually from the block pool or disk partition up). It stores structured data and unstructured data natively in a very unique fashion.

Comment Re:How about this (Score 1) 66

Working on it. And unlike the "storagedude", I do have the expertise to implement it. I have it about half implemented so far. I have created an object store where every object can have lots of attributes or tags attached to them. Unlike extended attributes, you can actually find things based on them quickly. For example, I can create a container and put 100 million of my data objects in it (photos, mp3s, software, documents, etc.) and find anything and/or everything in just a few seconds. If I had 10 million photos and they all had tags attached, I could find the 50,000 photos of my family vacation in Hawaii in 2010 in just a couple of seconds. Check out the video of my demo at

Hotels are tired of getting ripped off. I checked into a hotel and they had towels from my house. -- Mark Guido