CurtMonash - Slashdot User

Submission + - What's the best way to scale out Postgres? (dbms2.com)

Submitted by

CurtMonash

on Wednesday July 29, 2009 @12:17PM

CurtMonash writes: "I have a client building a new app that will peak at several million updates per hour to a 1-2 terabyte database. They don't have much budget, and they don't want to use MySQL. What are their options? They tried EnterpriseDB's GridSQL scale-out technology, but quickly discarded it because it didn't support the latest PostgreSQL releases. I put up a blog post with this question, asking about software and design options such as Hibernate and hand scaling. But the early responses have focused on SSDs. Also on the table are ACID-compliant in-memory DBMS like the Groovy SQL Switch or VoltDB, but those are immature products, and obviously require a lot of silicon. PostgreSQL has the benefit of being pretty mature, with some SQL features the client really values (e.g., windowing, geospatial datatype). What are the most effective and proven ways to scale out PostgreSQL?"

Comment Re:No. (Score 1) 503

by CurtMonash on Wednesday July 15, 2009 @01:22PM (#28705379) Attached to: Cats "Exploit" Humans By Purring

It's not an either/or. Cats behave towards us in some ways like they do toward their mothers, in other ways otherwise.

My cat treats me largely as his butler -- opening doors, fetching beverages, and so on. Perhaps he's been reading some P. G. Wodehouse.

Submission + - Two dangerous kinds of blog comment spam (monashreport.com) 1

Submitted by

CurtMonash

on Tuesday June 16, 2009 @06:13AM

CurtMonash writes: "As the owner of several WordPress blogs, I get over 100,000 spam comments per year, of which Akismet lets through a delightfully tiny fraction. Most of that spam seems to either be selling various (probably shady) products and services, or just attempt to transmit "link juice" to other sites for SEO purposes. At least two forms of comment spam, however, are more sinister than that.

First, some comments — especially ones focused on obscure "long-tail" keywords — try to direct visitors to actual malware delivery sites. I started seeing examples of those back in 2007; they're still coming in with high frequency. Second — and so far I've seen a single example of this one — I just got a comment containing a brazen offer to provide website-attacking services. Prices range from $25-70, depending on the duration of the attack, and the spammer promises "On average the data, ordered the site falls within 5 minutes after the start."

Aw shucks. I long for the good ol' days when spam just promised payday loans I could use to buy Viagra to enhance my enjoyment of free porn sites ..."

Submission + - MMO games use very strange database designs (dbms2.com)

Submitted by

CurtMonash

on Monday June 15, 2009 @08:26AM

CurtMonash writes: "The technology of MMO RPGs (Massively Multiplayer Online Roleplaying Games) is interesting and even groundbreaking on multiple levels — graphics, AI, networking/security, and more. But one areas where it's downright funky is in database management. Nominally, the major MMOs seem to use recognizable database technology — MySQL (World of Warcraft, Second Life), Postgres (Everquest), SQL Server (Guild Wars), StreamBase (also Second Life), and so on. But what they do with it is another matter. In Lord of the Rings Online (LOTRO), for example, minor database-value errors are only fixed in major quarterly releases, suggesting a rather cavalier attitude toward database integrity. Authentication servers go down all the time too, on what probably is a SQL- rather than LDAP-based system. Meanwhile, Guild Wars opts out of conventional database architecture altogether, and just saves character state in 10K-30K BLOBs.

Perhaps these are actually good technical decisions for the MMO developers to have made. But I can't help thinking that a little bit of enterprise IT savvy would save MMO developers a lot of embarrassment and aggravation, and make their gamers happier as well."

Submission + - Hospital turns away ambulances after EHRs go down (networkworld.com)

Submitted by

CurtMonash

on Thursday June 04, 2009 @03:07AM

CurtMonash writes: "The Indianapolis Star reports that Tuesday Morning, Methodist Hospital turned away patients in ambulances, for the first time in its 100-plus history. Why? Because the electronic health records (EHR) system had gone down the prior afternoon — due to a power surge — and the backlog of paperwork was no longer tolerable.

If you think about that story, it has a couple of disturbing aspects. Clearly the investment in or design of high availability, surge protection, etc. were sadly lacking. But even leaving that aside — why do problems with paperwork make it necessary to turn away patients?

Maybe the latter is OK, since there obviously were other, more smoothly running hospitals to send the patient to. Still, the whole story should be held up as a cautionary tale for hospitals and IT suppliers everywhere."

Submission + - Facebook's 2 1/2 petabyte Hadoop-based warehouse (dbms2.com)

Submitted by

CurtMonash

on Tuesday May 12, 2009 @01:03PM

CurtMonash writes: "Facebook has a 2 1/2 petabyte data warehouse, running on Hadoop/Hive rather than a database management system. I already mentioned this in a recent story, "Web Analytics Databases Get Even Larger"*, but subsequently I've gotten a lot more detail from Ashish Thusoo and Joydeep Sen Sarma of Facebook. Highlights include:

Facebook operates a single cluster of 610 nodes, running multiple Hadoop MapReduce jobs at once.
Hardware only costs $2000-$4000/node, versus the $10,000+/node that might be required for a DBMS.
Hive implements some basic SQL functionality over MapReduce.
Reliability is so-so. Long-running queries fail more often than ETL — which actually is ELT — because getting the data in the first place is more important than having query results be current to the nearest 15 minutes.
The most rapidly-cycled queries are re-run approximately hourly.
Facebook is really making serious extensions to open source Hadoop code.

*Sorry about the misspelling of "ever"!"

Submission + - Web analytics databases get every larger (dbms2.com)

Submitted by

CurtMonash

on Thursday April 30, 2009 @06:47AM

CurtMonash writes: "Web analytics databases are getting every larger. eBay now has a 6 1/2 petabyte warehouse running on Greenplum — user data — to go with its more established 2 1/2 petabyte Teradata system. Between the two databases, the metrics are enormous — 17 trillion rows, 150 billion new rows per day, millions of queries per day, and so on. Meanwhile, Facebook has 2/12 petabytes managed by Hadoop, not running on a conventional DBMS at all, Yahoo has over a petabyte (on a homegrown system), and Fox/MySpace has two different multi-hundred terabyte systems (Greenplum and Aster Data nCluster). eBay and Fox are the two Greenplum customers I wrote in about last August, when they both seemed to be headed to the petabyte range in a hurry. These are basically all web log/clickstream databases, except that network event data is even more voluminous than the pure clickstream stuff."

Submission + - MapReduce can't keep up with MPP DBMS (networkworld.com)

Submitted by

CurtMonash

on Tuesday April 14, 2009 @03:31AM

CurtMonash writes: "Monday was a bad night for the MapReduce advocates. First, famed MapReduce skeptics Michael Stonebraker and David DeWitt released a series of benchmarks that suggest MPP database management systems far outperform Mapreduce (specifically Hadoop). I piled on by posting some thoughts from even-more-skeptical eBay, which thinks MapReduce is 6-8X slower than MPP database managers for comparable tasks.

That doesn't mean MapReduce advocates need to jump off of a ledge. Much of what these benchmarks show is the should-have-been obvious point that MapReduce shouldn't be used to replace DBMS for tasks DBMS are good at. MapReduce applications tend to be concentrated in four areas:

Text tokenization, indexing, and search
Creation of other kinds of data structures (e.g., graphs)
Data mining and machine learning
Data transformation

and the benchmarks didn't really speak to any of those. But some of those areas may equally fall victim to the "Don't reinvent the wheel argument."

MapReduce is surely an appealing paradigm for lightweight, reliably-parallel programming. At least for research into parallel algorithms, it has much to recommend it. But whether MapReduce will play a major role going forward in production use seems at this point to still be an open question."

Submission + - Self Replicating Machines In Your Home - Seriously (singularityhub.com)

Submitted by

Singularity Hub

on Sunday April 12, 2009 @12:26PM

Singularity Hub writes: "Did you know that for about $500 you can have a machine in your living room that can print out a 3D replica of any object from a CAD drawing on your computer? You can use this machine to make door handles, dolls, cars, hooks...anything! This machine is called a reprap, and amazingly the specifications for the machine are completely open source, completely shareable and modifiable by anyone in the world. There is a worldwide community of volunteers working feverishly to support you and anyone else to troubleshoot and improve the machine. Most interesting of all, the reprap is ultimately designed to self replicate all of its parts, bringing us within tantalizing reach of a long envisioned era of self replicating machines."

Submission + - Twitter gets slammed by the StalkDaily XSS worm (networkworld.com)

Submitted by

CurtMonash

on Sunday April 12, 2009 @01:25AM

CurtMonash writes: "(This is a better version of what I just submitted, based on a more careful reading of the comment thread to my first blog post. Sorry for the dupe.)

Twitter was hit Saturday by a worm that caused victims' accounts to tweet favorably about the StalkDaily website. Infection occurred when one went to the profile page of a compromised account, and was largely spread by the kind of follower spam more commonly used by multi-level marketers.

Apparently the worm was an XSS attack, exploiting a vulnerability created in a recent Twitter update that introduced support for OAuth, and created by the 17-year-old owner of the StalkDaily website. Most of the details can be found in the comment thread to a Network World post I put up detailing the attack, or in the post itself. By evening, Twitter claimed to have closed the security hole."

Submission + - Twitter slammed by StalkDaily XSS virus (networkworld.com)

Submitted by

CurtMonash

on Sunday April 12, 2009 @01:10AM

CurtMonash writes: "Twitter was hit Saturday by a virus that caused victims' accounts to tweet favorably about the StalkDaily website. Infection occurred when one went to the profile page of a compromised account, and was largely spread by the kind of follower spam more commonly used by multi-level marketers.

The virus seems to have been an XSS/javascript attack, probably exploiting a vulnerability created in a recent Twitter update that introduced support for OAuth. Much of the diagnosis occurred in the comment thread to a Network World post I put up detailing the attack; the post itself contains page source and other particulars. By evening, Twitter claimed to have closed the security hole."

Submission + - Electric grid collapse -- two disaster scenarios (networkworld.com)

Submitted by

CurtMonash

on Monday March 23, 2009 @12:16PM

CurtMonash writes: "Nightmare scenarios are emerging right and left that could lead to the catastrophic collapse of the electric grid. The worst comes via the New Scientist, which suggests that a giant ball of plasma could destroy (almost) every transformer in North America. Medicine, food, and in some cases clean water would rapidly be unavailable, not to mention internet access. Tens of millions of people — starting with couch-bound geeks — would surely die. And by the way, just such a ball of plasma hit the Earth in 1859. More mundanely, next generation smart electric grids seem to have a lot of security vulnerabilities that nobody is in any hurry to patch, and unpleasantly destructive overloads could conceivably occur just via those."

Submission + - Star Trek:TNG actors go boldly into cyberspace (networkworld.com)

Submitted by

CurtMonash

on Friday March 13, 2009 @10:01AM

CurtMonash writes: "Three Star Trek: The Next Generation actors have ventured boldly into social media, engaging fans via Twitter and blogs alike. LeVar "Geordi" Burton recently offered Twitter followers first crack at tickets to his new play, and blogged a declaration that social media will be a large part of his life in the future. Brent "Data" Spiner is running an ever-morphing contest for the right to write his 500th tweet. And of course Wil "Wesley" Wheaton has a huge presence in cyberspace. How big is this? Well, Wheaton, Burton, and Spiner have more than 226,000, 122,000, and 46,000 Twitter followers respectively. Even more remarkably, Burton and Spiner both joined Twitter just this year. But it is still somewhat possible to engage these guys on a one-on-one basis, if you happen to catch their interest or mood."

Submission + - Brevity is the soul of 140-character wit (networkworld.com)

Submitted by

CurtMonash

on Monday March 09, 2009 @11:06AM

CurtMonash writes: "The New York Times noted today that Twitter is a fine venue for humor, but asked how to separate wheat from chaff. I offered an answer. It turns out that many Twitter users already hand-curate what amounts to one-liner joke lists, on their "Favorites" pages. My own Twitter favorites list, for example, contains a couple dozen witticisms, in a variety of genres, from popular culture, classical culture, rural, or techie to the just plain bawdy. But seriously, folks, there's a moral to this story: You find almost everything on the internet, including humor, in the same way — by following what amounts to a chain of recommendations from people, especially ones you have at least a little bit of a relationship with. That was true in the ancient era of chain-letter joke lists; it's just as true in the Brave New World of Twitter and YouTube; and it probably will be true in the next few generations of technology as well."

Submission + - Celebrities are gathering on Twitter (networkworld.com)

Submitted by

CurtMonash

on Saturday February 21, 2009 @02:29AM

CurtMonash writes: "Twitter got started as a kind of clubhouse for tech bloggers such as Robert Scoble, but it is increasingly being populated by celebrities of all kinds. People of some notoreity who seem to be posting to Twitter themselves range from athletes Shaquille O'Neal, Steve Nash, and Lance Armstrong to actors Wil Wheaton, Brent Spiner, and kinky-porn idol Madison Young. (By way of contrast, the Twitter stream in the name of Barack Obama is clearly run by flacks, while Britney Spears's is somewhere inbetween.) And they occasionally interact with "regular" Twitterers — Shaq invited fans to meet him at a diner, and Wheaton warned a viewer away from an unfavorite movie role."

Slashdot Top Deals