Welcome our new underground martian overlords!

I'm saying a database should be what we commonly interact with, instead of a lower level, limited conventional file system of directories and files. If you don't like interacting in this manner, you can still have a simple directory/file like layer on top of that, and it should work just as well if not better than existing systems. For compatibility I see no reason you couldn't replace the entire filesystem in a current computer with a database and have it still work the same on the surface, while using a database underneath, and then you'd have the option of accessing the database directly to do things you couldnt easily do with a simple filesystem.

After the discussion so far, I have realized that clearly there is still going to be some kind of low level filesystem to handle the physical formatting of the media, etc. Also there would need to be some sort of way of saying "I want this data to be physically on this particular hardware". But I still think dealing with a simple file/folder structure is going to be marginalized.

Seems like the real issue here is in defining what a database is.

Yes, technically, a file system could be considered a database. So too could m3u and mbox files be considered databases. Any collection of data is a database. But I think most relatively educated computer people would define a database more specifically than this. Generally, I think most people would consider a database to be a set or sets of data with accompanying index(es) used to quickly locate individual bits of that data.

m3u, mbox, etc would not be databases. mbox in combination with *.msf file, could possibly be considered a database.

Getting back to my two examples, which in hindsight are not the clear cut examples I thought they were (and thanks again for the wonderful argument, I love to argue! :)

I am sure Google uses some kind of a database, even if its not BigTable, for Indexing (note this word) the web and providing something to run search Queries (again note this word) against. I am no expert on Google but I am pretty sure when they index the web that they are not storing all this information into a huge, complex, hierarchical file structure. When searches are requested by users, I'm pretty sure they are not somehow mapped to a directory path where the file is the web page they are looking for.

They have a database (indexes to search against and a snapshot of the websites, minimum) of some kind that is used for their service. That database is of course stored physically using GFS.

mbox is a flat file containing lots of emails. The mail clients make little use of filesystem features for sorting, organizing or locating emails, though I believe most of them will have an mbox for each 'folder' you create, if you create any. Its not using the filesystem any more than, say, MySQL does when it creates a file for each table.

Oracle has databases that store directly to a harddrive, but I imagine even then, there's probably a simple filesystem of sorts underneath that handles the low level formatting.

I still stand by what I said. 'Everything' will go to 'databases' at the 'human level'. Obviously there's still going to be a filesystem at the lowest level. Just like there's still a "DOS" (not MS-DOS, just DOS in general) but nobody thinks about it anymore. Just like there's a "BIOS" of some sort, but nobody really thinks about it much..

Yes, mbox. A very modern, high performance email format...

so, mbox stores all the emails in separate files? And when you want to locate some old email, the complex file/directory structure that it uses to store and sort the emails allows you to search for an email quickly?

When I gave up using thunderbird in favor of gmail, one of the reasons I did so was the fact that gmail could search for and locate emails instantly, where thunderbird would take a considerable time to find something, with poor results. Do you know why this is? Because mbox sucks, its a flat file containing all emails, and databases are much better for this sort of thing. Thunderbird developers have discussed the limitations of mbox yeas ago. Not sure what the result was, but databases like Sqlite were considered.

Now, thunderbird, and I'm sure all the rest of the email clients made in this decade, create and store indexes to find emails. Those indexes in combination with even the archaic mbox 'format' constitute a database. The filing system does nothing more than save these two files to physical media.

I think everyone equates "database" with "MySQL" or more generally "Relational Database" but there's lots of other kinds of DB systems, like the recent 'NoSQL' trend (including Google's BigTable), and these are showing up everywhere.

http://en.wikipedia.org/wiki/BigTable

Yep, they use GFS, which is not a database, like FAT32 and the rest.

And on top of that is Big Table.

Which is a database.

Of course Big Table is relatively recent, but if you go back and read the original research paper presenting the Google search engine (pretty easy to locate using, of course, Google) you will note that it mentions 'database' more than a few times. Granted they've implemented their own highly optimized specific database instead of using, say, an existing relational database.. but its still a database at a much higher level than a file system.

'Filesystems' of course won't go away, you still need to handle the low level formatting of the media, do error handling, etc. But this will become a transparent thing most people generally dont need to think about.
 

You can have rigid hierarchies in a database too.

strict logical reasoning is actually quite the thing with database design, being based on mathematical set theory and such. You might not know this to look at the typical database design, but thats only because most people are idiots and shouldn't be designing databases (sorry, I'm at work, and I'm dealing with a horrid database design..).

I think everyone is missing the point, that a database oriented system should be able to do everything a typical file/directory based system can do, and more. If you need to have a simple file/directory abstraction on top of it, then you can have that.

The bigger picture though is the idea that you dont even need to be concerned about 'physical' files in the first place if everything is just a chunk of data in memory. No more loading/saving/copying. Sure you have 'document' concepts, like 'email' or 'letter of resignation' or 'picture of my cat' but these dont have to be explicitly loaded/saved.

You don't "load" an email or "save" it (typically, i know you can if you need to) you just create it and read it, the computer handles saving it, sending it, bringing it back, etc. I dont think there's an email program in existence that actually saves emails as files, or arranges them in physical folders on your harddrive. Personally I cant stand any email clients anymore because with google mail I can always search for a find an email instantly.

I like the idea that a word processing document is not an encapsulated blob but rather should be a connected arrangement of different bits of data stored optimally, ie, some text here, an image there, etc. We have been heading in this direction for some time now. Look at web pages.. lots of little bits. And look.. most web pages are stored in databases, not files.

I don't have all the answers.

I never said that you, or joe sixpack, should have to understand what a database is or how to use it. I was thinking about how the computer works internally. For you, or mr sixpack, it would seem to work largely if not exactly the same as it does now. However you'd also then have additional options if you cared to use them (in addition to the fact the system could potentially work faster and more efficiently even if you didnt).

Just like you don't have to understand that Google and pretty much every website on the net uses a database to store and find your stuff.

Also would like to note that you are talking about Joe Sixpack but also referring to unix commands as if the two are in any way related.

Let me guess. Are you also against "hiding" the computer's internal workings behind "icons" and "windows" as well? Or is it the other way around, that having icons and windows is a 'retardisation' and we should all be working with more expressive and simple shell commands?

Frankly your viewpoint is confusing to me. But I appreciate the argument, I love a good argument!

Mark my words: standard file systems will go the way of the dodo. They will be replaced (and indeed this is already starting to happen) by database oriented designs. The hard part is not in making such a system, or in making it just as easy to use; the hard part is the transition to such systems. Just like electric cars and the infrastructure to support them (or other alternative fuel systems). The internet though, is making that happen.

I didnt say they suck, just that they can be annoying too. They've served us pretty well for some time. But maybe its time for something new.

A neat, rigid hierarchy has its limits. Just ask biologists who sometimes have trouble figuring out just where to put some odd species into the biological classification tree.

All I'm saying is that computers can handle more complex organizational methods, so why not use them?

a) what do you think file names and directory paths are? Some bit of data you are looking for in a database can also be tagged with "metadata" that is analogous to a filename and path.

b) I can't tell you how many times I've seen somebody (including myself) try to find something they know they saved somewhere, in a folder, but cant remember the exact path, or for that matter thought they'd make it easy and just dump everytning into /downloads but now they have to remember the file name and/or the approximate date they saved it, and starting sorting by date or name trying to find it..

In our existing world, file names and nested directories are really freaking annoying too..

Let me pose this for you: do you think Google stores things in a big organized hierarchical file structure, just like a human would do it with paper, manila folders, file cabinets and such? Or maybe they use, I don't know, some kind of "database" or something? Considering the billions of times per day that people are looking for things on Google, I would say, databases are pretty good at finding stuff.

I think you must have missed the point I was making about the fact that you can easily build something just like a plain file/directory system on top of a database underneath (in fact, I believe, some filesystems in fact do just this, though I havent kept up on modern file system design..). But a database built on top of a filesystem uses very little of the functionality of the file system and in fact the file system is probably a hinderance in general.

Lets make the inevitable car analogy!

I think its like combustion engine cars vs electric. Electric is way simpler, far more efficient, provides better torque and a wider performance profile, etc etc. In fact electric kicks the shit out of combustion in every way except for the battery requirement. And thats just a matter of coming up with lighter batteries that hold a bigger charge and can be charged/refueled/replaced quickly and cheaply. And that *will* happen in time.

Same thing with computers. It will happen. Eventually. Just look at the internet. Its already backed by databases primarily, its distributed, its parallel. All that will make its way down to the smallest computer. If you think URLs are like files think again, how many URLs map directly to a file? How many URLs are connected to their data through hash tables in a fixed hierarchical storage structure?

I, for one, welcome our new file-less overlords!

My first thoughts when reading this (which is not to discount the fact that I've thought about the subject many times before, including concepts like the resource fork in older mac systems, etc) are:

Why have files at all? Files are only there as abstractions because we are familiar with the physical concept of files and documents.

I'm sure I'm not the only one to think down these lines. Are there "files" in the volatile memory of your computer? Generally, no.. there are blocks of memory, with address pointers and such to chain them together in way they can be found. First of, instead of translating back and forth between memory and disk files, why not just have one huge addressing space and store everything in that, and let the system decide what to move to persistent storage. The hardrive, or whatever, is just a big cache/persistent store that you rarely think about (of course you'd want to be able to 'hint' the system about what should be persisted right now). Once you've done this, there are no files anymore.

Of course, you need to be able to locate things. And we have tools for this. They're called databases. Whether its a relational SQL database or a key-value store NoSQL type or even something else, they don't use the file analogy. The actual persisted storage unit would probably just be a database directly stored to the media using whatever formatting was optimal (rather than stuffing a database into a file..).

Instead of opening a file, you pose a query and the database finds the data for you. Really, not much different than now. open('filename','r') is really just a query too and could still work the same way in such a system for compatibility.

There's a lot of work to get to where this would be efficient I think, and requires changing how we think about some things. Letting go of preconceived notions of physical files etc. But that's happening more and more. Things are going all digital. It may be 10 years or more but I think it will happen...

At some point there will be no more "files" or "directories", there will just be information, and questions about the information.

.. as long as he doesn't cut the military budget.

We'll need the army when the american people revolt and martial law is instituted.

Ah, good. It changes everywhere! Good Slashdot! Now I dont have to worry about any old postings containing a reference to a screwed company I used to work for.

I am a loyal employee as well, generally. But in my years of working, I've been royally screwed three times in return for my loyalty (including just recently). But more often then not, I get handed opportunities because the people I work with want to continue working with me in some fashion. Being fair, loyal, and producing good work does pay off.

But, don't be a sucker either. Any reasonable person will understand if you have to make a hard choice because its the right choice for you and your family. I've done that, and it always plays out well. Be fair, make some compromises if needed, but remember whats really important in life.

Now if I can just remember how to change the signature below..

And, what, you expected me to READ the article first? Pshaw! This isn't the slashdot I know and love if you actually expect me to read the article and be informed first...

I'm sure I'm one of thousands of folks thinking that how to glue together chips must be the least concern, and how to dissipate heat must be the highest?

The only thing I can think of that makes the adhesive important would be how well it holds up under heat, so maybe thats why its hard to do?

I imagine such a "brick" of silicon would probably have to have active cooling build into it, such as etched-in heat pipes or even some kind of micro fluid cooling system. Thats where the interesting stuff is happening.

I mean, really.. glue.. how exciting is that?