Well you misunderstand scalability. Scalable DOES mean you can just throw more hardware at something, as you said. But you omit the part the scalability has limits. There's always a bottleneck, so no app is infinitely scalable. You take your best guess at what its intended audience will be and you proceed form there.

You can say my app's been designed so that it will scale when n is between 1 and 100,000 for instance. Or that it'll scale between 1 and 100,000,000,000,000. But the app you design for the later will be very different internally and take a lot more time to design. Most apps start out working only for small n because most apps will never get to big n or if they do, they do so gradually and give developers time to deal with it. Twitter was cursed by its own success, they didn't know it'd be as popular as it turned out.

Let's look at some specific technical details.

For most sites a single master DB server with slaves for reads is fine. Most apps out there don't need multi-master capabilities and thus implementing that would be a waste of time. In fact, most web apps you see out there are designed to run on a single DB server for reads and writes.

When Twitter started that's what they had and it worked. Then twitter got unexpectedly popular, and at that point in time Rails did not support multiple DB connections per instance, so sharding wasn't really possible. Days after they announced this problem it was fixed so it's no longer an issue.

We could speculate on problems they've had since then, but that would just be speculation. They've said repeatedly however that their stack's scalability issues have been in the non-rails parts (like the DB). Since they're the only ones who actually know what's going on, I'm inclined to believe them.

Let's look at some other social app failures,

An example would be myspace, which had horrible problems scaling, even though that was written in PHP which you consider to be super scalable. Or we could look at friendster, which was written in enterprisey java, which also had huge scaling problems. It's not the tool, it's how you use it.

Additionally, when you scale real world systems you just may find out the bottlenecks aren't where you thought they'd be during the design and testing. Runaway successes like twitter are vulnerable to this fact.

If you want to fault the Twitter engineers you really can only fault them for bad judgement in estimating the size of their user base. I don't think you know what Rails actually is. It's just helpers for YOUR code. Ultimately, a rails site is mostly your code, and your architecture.

Your ad nauseum argument says nothing, but since some moronic people with mod points are out there I may as well respond.

You haven't said anything specific about rails. There's tons of successful rails deployments out there, just because twitter's engineers suck at what they do doesn't mean the tech they use is bad. Tell me WHY does rails suck? What part of it is causing all these problems?

Also, Twitter isn't just rails, from all accounts the problems they have are from other parts of their stack. Again, not the tech used there, but their inability to deploy and set it up properly.

Also, If you've ever read about the architecture of Facebook you'd realize it's an extremely complex application. It's not just a simple LAMP stack, there's all kinds of caching and queing, with large hadoop clusters etc. Facebook is a HARD PROBLEM with a complicated solution. Luckily for facebook, they engineered it correctly. Those same tools if used improperly would not work.

Furthermore, your assertion that the data is easy is wrong. EVERYTHING IS EASY FOR SMALL N. The problem isn't the content, it's the quantity of messages. You may as well have said a man who eats 1000000000 jelly beans did something far easier than a man who ate a large burrito.

In short, don't hate the tool, hate the guy who's not using it right. Also, if you're going to critique something, be knowledgable and specific about it, don't just say X sucks because it sucks. Your ad nauseum argument says nothing.

Ummmm except that they handle a lot of little tidbits of text. Twitter isn't some chintzy simple site, they handle a HUGE volume of data. Their scalability problems come from their database layer (or rather their inability to setup a stable database layer) not Rails.

I'm punching the next person who posts to this that it doesn't scale because of rails in the face. I'm sick of people who don't understand the HUGE distinction between scalability and performance.

I get 950 Mbps with iperf (w/o jumbo frames). Actual disk to disk, or even disk to memory transfer speeds are at 30MB/s. If you'd read the article I linked to you'd know that.

Actual protocols are a LOT slower than just slamming as many bits as possible through a pipe.

The days of network being faster than disk are over.

Gig-E speed is about 30MB/s in the real world. This is with a crossover cable, machine to machine. I've tested and verified this over a number of platforms, including expensive server systems.

Cheap terabyte single disks these days can do 80MB/s.

The only reason to go with raid is for redundancy, or better handling of simultaneous connections.

Read this for a more in depth analysis: http://www.codinghorror.com/blog/archives/000339.html

That USED to be true. It's not the hard drive, all the layers that get put in between when you access a disk over the network. Modern hard drives can easily do 60MB/s sustained.

For instance, I have a couple raid6 arrays which clock in at about 250 MB/s and 150MB/s natively. If I hook that machine up directly to another machine's ethernet port I only get about 30MB/s sharing the device w/ iSCSI. SMB and NFS yield similar results. This is true even though I can get over 900Mbps using iperf.

Sharing disks over gig-e sucks when you actually need throughput. It's great for when you just need to expand a SAN and speed is secondary. I've heard that bonding two Gig-e cards doesn't realize much of an improvement FWIW, so I assume latency is part of the reason it's slower.