Well, the storage cost per gigabyte-month is $0.15 and the storage cost for transferring a gigabyte outbound is $0.17, so if you download the stored data once per month then transfer is the dominant cost.(1)

Someone is always paying for the cost of storage and transfer. Before the person who owns the bucket would pay for both, but now they can make the accessor pay for the transfer. Amazon isn't offering either for free, so as long as they are making a marginal profit on storage and transfer, they aren't eating any loss.

(1) Both numbers are for the base tier.

In that case I agree with toddestan. The memory market is too competitive for that. I believe a bunch of memory manufacturers are hurting this year (and their stock is being pummeled) because their margins on flash chips are extremely small. The price differentiation will occur based on the performance of the firmware and controller. The memory is dirt cheap, but the manufacturers with good firmware (such as Intel) will be able to provide a price premium until other SSD manufacturers catch up.

Really? Due to the physical moving components of a hard disk, they tend to bottom out at $30. With smaller disks, you get diminishing returns because the price of the moving parts dominates. With SSD, however, the price scales fairly linearly all the way to the bottom. Applications that don't need much storage (such as netbooks or low end laptop/desktops) will move to SSD when the price SSDs for the necessary amount of storage is less than $30. If you only need 30 GB in a netbook, SSD is a more logical choice.

Also, it depends on how you quantify priced below regular drives. I assume you are talking about $/GB. However, if you look at $/IOPS (IOs per second), then Intel's SSD is way cheaper than any hard disk. SSDs are going to quickly take over database servers in 2009.

I posted this as a comment on the blog post, but I'm copying it here as well:

http://blogs.smugmug.com/don/2008/06/03/skynet-lives-aka-ec2-smugmug/

Outside of one instance where it launched 250 XL nodes, it seems to be performing pretty well. Their software takes into account a large number of data points (30-50) when deciding to scale up or down. It also takes into account the average launch time of instances, so it can be ahead of the curve, while at the same time not launching more than it needs.

Contemporary libertarians remind me of children who never learned to share.

I consider myself a libertarian and I donate to philanthropic causes. However, I believe that the free market is a better way to organize the distribution of the my wealth and that I have a better idea of where to donate my money than the government. You want to take my money and give it to people who you believe are worthy. I want everyone to keep their money and (optionally) give it to organizations they individually believe are worthy. Big difference there, see?

I should have been more specific/clear. If you read do a full read of a terabyte disk a dozen times, you are likely to see an unrecoverable read error:

"Typically, [Unrecoverable Error Rate (UER) for read operations] will be 1 per 10^14 bits read for consumer class drives and 1 per 10^15 for enterprise class drives. This can be alarming, because you could also say that consumer class drives should see 1 UER per 12.5 TBytes of data read."

That quote is from a Sun blog that has lots of information about Mean Time To Data Loss. His other posts are interesting as well.

Almost, but not quite. MapReduce has a slightly different format than just map() and reduce(). Here is the signature of map and reduce from a theoretical functional language:

map(): A* -> B*
reduce(): B* -> C

Whereas in MapReduce:

map: (K, V)* -> (K1, V1)*
reduce: (K1, (V1)*)* -> (K2, V2)*

I think that is mostly accurate. Read more accurate/detailed report in MapReduce revisited[PDF].

The individual functions map and reduce are quite standard. The innovation here is the systems work they've done to make it work on such a large scale. All the programmer needs to worry about is implementing the two functions, they don't have to worry about distributing the work, ensuring fault tolerance, or anything else for that matter. That is the innovation.

They mention in the article that if you try and sort a petabyte you WILL get hard disk and computer failures. Hell, you can only read a terabyte hard disk a few times before you encounter unrecoverable errors. The system for executing those maps and reduces is what is important here. The important parts are in the design details, such as dealing with stragglers. If you have 4000 identical machines, you won't necessarily get equal performance. If a few of those machines have a bit flipped and started without disk cache, they might see a huge decrease in read/write performance. The system needs to recognize this and schedule the work differently. That can make a huge difference in execution time. If you graph the percentile complete of a MR job, you'll often see that it quickly reaches 95% and then plateaus. The last 5% may take 20% of the time, and good scheduling is required to bring this time down.

But like I said, the innovation isn't in the idea of using a Map and Reduce function, it is the system that executes the work.