Forgot your password?
typodupeerror

+ - Ask Slashdot: Do you want a local object store (i.e. a flat file system)?->

Submitted by
DidgetMaster
DidgetMaster writes "Object stores have been around for a while now. For example, Amazon S3 storage is a set of buckets in the cloud in which you can store millions of files as "objects". Local file systems have been around much longer. For nearly 50 years now, we have been stuck with the traditional hierarchical file system storage model (e.g. a tree-like structure of folders or directories). Both systems make it easy to store a ton of data and to find a single item very quickly if you know its unique ID (full path for files, key for objects). But both systems are terrible at searching for all data that have certain features. If you have 2 million files on a file system volume and you want to find all pictures (*.jpg, *.png, *.ico, etc.), then it takes forever to scan the whole system looking for them. Cloud systems are not any better at search. Adding extra metadata (e.g. tags or extended attributes) can help distinguish one file or object from another, but searching for things based on those is even slower. "Find all documents where Author=John" is only fast if all the metadata has been collected and stored in a separate database, otherwise go to lunch while you wait for the results.
The Didget Management System wants to change all that by introducing a new object storage model designed to replace file systems. A Didget (Data Widget) is like a file that can contain any unstructured data stream up to 16 TB in length, but it is also like a row in a NoSQL database table where lots of searchable structured tags can be attached to it. Structured and unstructured data can be stored side-by-side within the same container and both types can be returned as the result of a query. Note: this is NOT like other indexing systems like Spotlight or Windows Search. See the 5 minute video demo at http://screenr.com/XV17.
Is the ability to instantly find "All photographs taken in Hawaii in 2011" when there are 100 of them among 5 million other pieces of data, enough for you to want to replace your file system with something new?"

Link to Original Source
This discussion was created for logged-in users only, but now has been archived. No new comments can be posted.

Ask Slashdot: Do you want a local object store (i.e. a flat file system)?

Comments Filter:

Advertising is a valuable economic factor because it is the cheapest way of selling goods, particularly if the goods are worthless. -- Sinclair Lewis

Working...