Comment Re:OK, tell the truth (Score 1) 295
I was thinking of something a little more "Being John Malkovitch" personally.
I was thinking of something a little more "Being John Malkovitch" personally.
That article you linked to is one of the craziest things I've ever read. RDBMSs are powerful enough to do anything -- but so many engineers are too lazy to learn proper SQL.
Actually there are plenty of engineers who know not only SQL but other query languages too! Proper SQL doesn't help you get extra parallelism in a write-intensive environment.
If you need to handle more transactions or queries, buy a bigger box. You can have hundreds of TB in a nice enterprise-grade server. If you HAVE to run your data across multiple machines, just spend some time and actually *think* about what data you want where, and then write a little code to send it to the right database.
And all that disk does you how much good when you need to read ALL the data into RAM to run a query? Oh, right.
Even Google used a ton of MySQL boxes for a long time to deliver their searches. Scale-out architecture is a lot more of a lie than its proponents make it out to be, check out this post on CodingHorror.
Emphases on "used" rather than "currently uses." If you've got more than a few TB worth of data scaling out is really the only good way to go. And Jeff Atwood takes a simplistic look at licensing fees (and assumes that you're not using FOSS).
Eventually, all these new "databases" will need to actually be used in a *real* environment, and they'll have transactions, SQL support, indexing, a good UI, and everything else. All MS or Oracle need to do is a little tweaking to make their multi-box configurations more robust, and they'll crush everything else.
You know what? You're right. In fact, Google and Yahoo are currently in the process of converting BACK to SQL right now. They did the whole scale-out thing and it totally sucked. While it was good enough to run the company and ensure that everything worked reliably and whatnot, it wasn't good enough to appease the automatic report generator machine. That and pretty GUIs are what really matter in life. Not simplicity, robustness, and CERTAINLY not scalability.
I couldn't agree more about databases being terrible for genetic data. I do some work in bioinformatics right now. Completely aside from the terrible code quality is the overwhelming desire to shoehorn all this stuff into databases because nobody knows any better. And when you're working with genetic data you don't need transactions, you don't need some kind of rigid consistency, etc. You just need something to hold all the data for you and take care of picking it up from the disk.
A college friend recently wrote a great article about the shortcomings of regular databases for a lot of the challenges the community faces today. It's available at http://www.roadtofailure.com/
For God's sake, stop researching for a while and begin to think!
