Submission + - It's Time to Replace File Systems (substack.com) 4
DidgetMaster writes: Hard drive costs now hover around $20 per TB. Drives bigger than 20TB are now available. Fast SSDs are more expensive, but the average user can now afford these in TB capacities as well. Yet we are still using antiquated file systems that were designed decades ago when the biggest drives were much less than a single GB. Their oversized file records and slow directory traversal search algorithms make finding files on volumes that can hold more than 100 million files, a nightmare. Rather than flexible tagging systems that could make searches quick and easy, they have things like 'extended attributes' that are painfully slow to search on. Indexing services can be built on top of them, but these are not an integral part of the file system so they can be bypassed and become out of sync with the FS itself.
It is time to replace file systems with something better. A local object store that can effectively manage hundreds of millions of files and find things in seconds based on file type and/or tags attached is possible. File systems are usually free and come with your operating system, so there seems to be little incentive for someone to build a new system from scratch, but just like we needed the Internet to come along and change everything we need a better data storage manager.
See Didgets for an example of what is possible.
It is time to replace file systems with something better. A local object store that can effectively manage hundreds of millions of files and find things in seconds based on file type and/or tags attached is possible. File systems are usually free and come with your operating system, so there seems to be little incentive for someone to build a new system from scratch, but just like we needed the Internet to come along and change everything we need a better data storage manager.
See Didgets for an example of what is possible.
