"...at maybe +2% cost." Now who is making up numbers? Something tells me that if this big boost in efficiency could be achieved with minimal cost, then that fact would be displayed predominately in the article. Since they didn't even mention cost anywhere, I am highly skeptical that it will be cheap.

Are you still considered a 'Climate Change Denier' if you think that the climate is changing, but are unconvinced that all the changes will result in catastrophic events? The climate has always changed and has been much warmer and cooler in the past. When did it get decided that the average temperature in 1900 (or pick any other date) was the ideal, and any change from it spells disaster? I find it interesting that the most alarmist people in the climate debate consider themselves progressive politically. I always thought that progressives believe that change = good while conservatives generally try to conserve the status quo (i.e. change is usually bad). Why is change with regards to climate always considered bad for progressives while every other change is considered good?

I wish that wasting their time got you pulled off their call lists. My response to any question asked by a telemarketer is "Hold on, let me check!" and I put the phone down and keep working, keep eating, keep watching TV, etc. They usually hang up within a minute of getting no response from you but they seem to call back later. If everyone did this (wasted even 1 minute of their time for every single cold call without buying anything), then the whole industry would go bankrupt in short order.

Actual link... https://www.youtube.com/watch?...

Maybe you would like this one then. A 5 minute video showing query times against a 7.4 million row, 22 column table (Chicago crime data) on Postgres v13 vs a new kind of database engine (Didgets).

https://www.youtube.com/watch?v=R6lhqeDhCms

Like many startups, they seem to have gone with the simplest solution that worked for the size of the data and the nature of their initial queries at the time; and then were surprised when the solution started to bog down once the size of the data grew beyond their estimates. This happens quite often in small businesses that still use spreadsheets to manage their data instead of a relational database. Once your data reaches a certain size or the scope of your queries expands significantly, the old solution may no longer work properly.

When I was developing my own database engine, I found it really hard to find reliable benchmarks for other systems. Each benchmark can be affected by a hundred different factors like configuration settings, indexes, or query plans. Like this blogger, I had to run a bunch of different kinds of queries using a variety of data sets against other DBs like Postgres, MySQL, and SQL Server before I could determine that my own solution was faster in most instances. I tried to make the competition as fast as possible by creating indexes and tweaking settings. Even then, I had to wonder if I was missing some important setting in each of those other DBs that made my benchmark tests 'unfair'.

I have a substack newsletter (Didgets.substack.com) where I post every few weeks. It is completely free for anyone to read so I don't think substack makes any money off my writing. I mainly use it to raise awareness of the new data management platform I am building (Didgets.com) but I try to make it general and interesting enough that anyone with an interest in data storage and processing would like reading it. I guess that substack is hoping that I will want to move to a paid subscription model once I have a ton of subscribers and they will get a cut of the fees.

Everyone knows that adding (4 + 4) takes twice as much power as adding (2 + 2). The real trick is knowing it was (4 + 4) when the power doubled instead of (3 + 5) or (6 + 2)? /s

A data value is often meaningless without a contextual framework around it. The number '71' for example is just a number. It is a data point, but without knowing what context it has, it is pretty meaningless. It is the current temperature? Someone's age? The amount in your checking account? Even if you know what it represents, it is often meaningless without comparing it to other values with the same context. If it is the temperature, what time of day or time of year was it measured? How far from 'normal' is the value?

The author is right that data often does not translate well between applications (even though he said it is useless between them, which is often not true). It can be really difficult to know if two data sets are in any way related. Even if you have two tables within the same database, a column called 'temp' in table1 might (or might not) be related to a column called (degrees) in table2. Are the values for the same location? Is one set of values measured in Celsius while the other is measured in Fahrenheit? How is each value related to other data around it?

I have spent my career dealing with data sets and building tools that help manage them. In order to share data between applications, there is a great need for better metadata that helps you understand context. I built a tool that helps drill down large data sets to let non-database experts do simple analytics on relevant subsets of that data. It is in open beta at www.Didgets.com, if anyone is interested.

This seems to be a common argument these days. If you want something to cease to exist, you just redefine what the word means then claim that the thing you don't like doesn't match your new definition and therefore it doesn't exist. In the meantime, you completely ignore what the general understanding of the word has meant to the majority of people over hundreds or even thousands of years. This is a real popular tactic in the world of politics.

'potentially ten times more common...' tells me nothing. If it were previously thought to be 1 part per billion and would cost a fortune to extract but now it 'might' be possible to find it in concentrations of 10 parts per billion but would still cost a fortune to extract, then how does that help us?

Kind of like your privacy and data in the 'cloud'. If you turn over your data or personal info to Google, Microsoft, or other cloud provider then they can do anything they want with it (regardless of what the current privacy policy might be).

The tagging feature of Didgets can be a very convenient way to organize data, but there is no requirement that every Didget must have tags attached. Anyone is free to attach whatever tags they like. Obviously, the easier you make it for users to tag things the more they will do it. I anticipate that a lot of the tagging will be accomplished automatically through things like AI in the future. Facial recognition software, for example, could look at every photo and automatically add tags on people and things it recognized in the picture.

The file type can be assigned through the API or the system can assign the type based on the file extension. There are a set of predefined types (e.g. .jpg = Image) but the user can specify additional ones. Obviously, not every file has a clear category so there is a 'Special' classification and everything that doesn't match any defined types can be classified as 'Unknown'. Even if you can only classify 90% of your files, that can really narrow down your search when you are looking for something specific. When importing existing files, the 'drop zone' where you drop the files or folders can have tags attached to it. Anything imported through that method has the specified tags automatically attached to it.

So, why has Microsoft been trying this for ages? Is it because there are some people who find traditional filesystems lacking in some way and they wanted to fix it? Judging by the comments so far, there are a lot of people who think filesystems are perfect the way they are and are offended by anyone who suggests otherwise. According to them anyone who attempts to change the status quo must be a moron and doesn't understand anything.

I never said that Didgets was a perfect system and it solves every problem imaginable. It is a huge project with lots of pieces. It might very well be that filesystems will eventually be replaced with something completely different from it, but will that ever happen if people refuse to even consider the possibility that there might be something better?