Amazon's New Storage Service

dlaur writes to tell us that today Amazon announced their Simple Storage Service (S3) allowing users to store unlimited amounts of data at $0.15 per GB paid monthly. From the article: "S3 was purportedly built to support both Amazon's own internal applications and the external users of the Amazon Web Services platform. That should be proper motivation to build a service that's fast and robust enough for mission critical use, yet flexible enough to support any storage task thrown at it."

Encryption

[grahamsz]

I wonder how long it'll take to build a backup solution that encrypts your data locally with a private key before sending it off to amazon. That way they wouldn't be able to look through it, and at 15c/month/gig it'd be pretty affordable for home backups

API proliferation

[gordyf]

It's good to see more sites adding APIs to their web services (Amazon already has other web services [amazon.com], as does Yahoo and Google of course). It's becoming even easier for "mere mortals" to link together new technologies to make innovative new systems, but I wonder if this reliance on third-party systems comes at a cost, perhaps to reliability or security?

Anyone remember GoogleZon?

[Nomihn0]

This is an interesting case of diversification. Amazon, no longer content to be the middle man e-tailer, is shifting it's weight into Google's territory with a service-based profit model. If this trend continues at Amazon, I have to wonder if Google will make a hostile bid for its newfound competitor.
 
  Here is a Link to EPIC [robinsloan.com], a speculative piece on the future of media, including the GoogleZon segment.

Could file storage services use this?...

[lux55]

Since this is nothing more than an API to access data, I wonder if this couldn't be used as the backend storage for existing file storage services, instead of paying for servers and bandwidth yourself...

This limits costs to storage actually used (at $0.15/GB which is a very fair rate), and bandwidth actually used. The cost that could add up is the bandwidth, where you'd obviously have to direct users to the amazon URL directly to avoid using bandwidth to get the file then to pass it on too.

Plus, at $0.20/GB of bandwidth, upload/download could get expensive still, with no cap on that cost. For example, 2,000 GB of bandwidth, which is bundled with most low-end dedicated servers nowadays (ie. even the sub-$99/mo. machines), this would cost you $400 from Amazon. That's pretty steep, and may be the limiting factor making it unfeasible for this idea. Interesting nonetheless.

Statistics can be your friend

[the ed menace]

In general you might think that the cost per gig you pay and the cost per gig that Amazon pays is similar. And you would be right. But to obtain high reliability you will pay more.

Amazon, or anybody who tries to build a large distributed storage service, can spread out the probability of disk errors over a larger set of users than you are able to do. The marginal cost to replace a disk that has failed, on a per user basis, is therefore lower for Amazon.

Moreover, the overhead to manage many disks does not increase linearly to the number of disks. Put another way, their per user cost to manage the disks is lower than you.

The cost equation is less about purchasing the storage than maintaining it through the inevitable failures over time. This makes the gigabyte-based usage cost very fair, since it is proportional to the rate of error. The access cost manages their bandwidth expense.

What I would like from a service like this is a pricing guarantee -- if they maintain the same pricing two years from now, it will be a ripoff given the diminishing cost of storage and bandwidth. It would be nice to have it pegged to some kind of disk/bandwidth industry index.

Backup Buddies?

[cgenman]

It seems like this new service would be best for offsite backup of prescious data.

However, it isn't all that cost-effective. A local disk is very cheap comparatively, but (as a friend of mine found out) if someone steals your computer, they steal your backup too.

Are there any services out there which connects people with reasonable connections over long distances to back-up eachothers data? I'd be willing to get a new 80GB drive and make it available via a private FTP server if someone else would do the same for me.

Or are there cheaper offsite solutions than Amazon's?

BitTorrent

[elrond1999]

Built to be flexible so that protocol or functional layers can easily be added. Default download protocol is HTTP. A BitTorrent (TM) protocol interface is provided to lower costs for high-scale distribution. Additional interfaces will be added in the future.

Amazon supports BitTorrent for the storage. Does that mean they run the tracker? Interesting way to save on transfer fees that :)

I wonder how Google Drive will compare to this

[FleaPlus]

It looks like Amazon managed to get their storage product out before the rumored Google Drive (TechCrunch article [techcrunch.com], Slashdot article [slashdot.org]). I wonder how Amazon's product will compare to Google's, whenever Google's is released. I'm particularly interested in seeing how Amazon and Google will end up competing with each other in terms of price and transfer speeds.

Re:Plus $0.20 per Gb transferred!!!

[OpticalPaul]

When Amazon lets me pay for the storage, and have other registered users pay the bandwidth charges (plus my profit) to access my content, then they'll have an interesting business. Unless Google beats them to it.

As it is, online remote storage with ongoing upload, download, and storage fees hardly seems interesting.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Backup

[scoutts]

Running .15 per Gig plus .20 per Gig transfer is not very economical for power users, nor very practical (upload speeds etc) This is pretty well covered. However..for the typical end user that can barely patch their machine, , may or may not understand to renew their virus definitions adn only use their computer to store digital pictures of their grandkids this is a good service. I often wonder how many grandparents lose their photo collection everytime a hard disk crashes (3-5 years at most) because i doubt more than 1% of home users run a raid array. Yep, better choices, flickr etc but which allows them to be hosted and send links to friends but nice to have a generic service. Amazon "catering to the unsophisticated". got a ring to it :)

Let's compare

[EmagGeek]

Let's take a 300GB System that needs to be backed up:

900GB at $0.20/gb = $180.00 transfer fee
Monthly at $0.15/gb = $135.00/month recurring charge
Weekly incremental of 30GB = $6.00/wk = $24/mo recurring charge

So $180.00 + $159/month = $2088 just for the first year, plus whatever you have to pay your ISP for abusive bandwidth charges.

Let's look at it from another perspective:

4 WD3200SD 320GB Raid-Edition SATA Drives: About $600
1 4-Port SATA Raid Card: About $250
Expected Lifetime: 5 years

So, buying a whole other raid-5 array to mirror your 900GB of stuff costs nearly $10K to store for 5 years on Amazon versus $850 to store locally. Hell, even if you were paranoid and replaced one of the hard disks every 3 months, you'd still be at less than half the cost.

I won't even get into which is more secure. If it's not on your site or some place you have physical control over, it is not secure.

Re:Terms Of Service

[morgus morphus]

Actually, a little further in the agreement they specify that they are only allowed to delete your data 30 days after terminating the agreement with you. So they certainly intend to give you safety against that (the details of how you get your data back after the agreement was terminated I don't know, but at least it's nice to see that they have recognized the concern).

The MPAA is behind this!

[Anonymous Coward]

Remarkable how the "Etherdrive" phenominon seems so closely linked with the "IP-VOD" Rumor-space. Here is the evidence:

Apple: .Mac capacity increases (big-uns) in Dec, iMovies in April
Google: Google Video in (Nov? Jan? Don't recall), GDrive any day now
Amazon: Movies-R-Us ambitions (like, last week), S3 Today

Conclusion, the MPAA will allow these guys to sell movie downloads as long as they keep part of the film on their own servers! How better to ensure you don't put those same vids they send you back up on Torrent than simply not to give you the whole file. When you want to fire it up, you get half from your hard drive, half from your network drive.

It's Perfect for the MPAA! The studios get the control they want, "The People" get the convenience they desire, and the cost gets pushed to the third-party vendors (who, conveniently already own complicated things like server-farms). Complete control, none of the hassle!

Of course, no one will be able to put up a fight when first the vendors, then the studios start pushing ads into the front of "your" movies to "cover the increasing costs of doing business". Remember when cable television and satellite radio had no ads. In a few years it will be just like going to the theater! Groan.

fees and limits

[MORTAR_COMBAT!]

so, $0.15 per GB/month storage, $0.20 per GB transferred.

questions: how do i put a cap on my storage (and more importantly transfer) so a runaway service doesn't screw me?

NOT a backup solution

[WPIDalamar]

From the terms of service...

2) You may make calls at any time that the Amazon Web Services are available, provided that you either: (i) do not exceed 1 call per second per IP address, or send files greater than 40K; or (ii) do not exceed the limits set forth in the Service Terms for a particular Service. If you build and release an Application, the stated limitations apply to each installed copy of the Application.

I use rsync.net for this already - encrypted, etc.

[Anonymous Coward]

I have been using a service just like this at rsync.net. It is a little bit different, but the concept is the same - online backup, redundant, etc.

The difference (I think) is that I can't imagine amazon.com will have theirs open for plain old ssh and ftp, which is exactly how I use my account at rsync.net.

I scp to and from it, rsync to it from cron, and have it mounted as a local drive over sshfs (and I can't say enough about FUSE/sshfs - I should submit a slashdot story just for them).

I think the price is comparable - the $2/gig I pay seems like a lot more, but I note you have to pay 20 cents per gig per transfer with Amazon, whereas with rsync.net I get unlimited transfer.

Allmydata

[P!Alexander]

I've tried Allmydata [allmydata.com] in the past and had some success. They use a peer-to-peer system to back your files up. The free plan requires a 10:1 ratio (you share 10 megs, you get 1 meg). Not bad if you have a lot of free drive space sitting there doing nothing. I think it uses bittorrent as its transfer method but I can't remember exactly how it works and the web page is short on details.

I just got the most recent version (1.3) and haven't played with it much but the last one I had a lot of trouble with, sometimes files wouldn't upload, it was hard to tell if the program was actually working or if it had died, and it seemed like it couldn't remember which files I had chosen to persistently back up.

Still in beta though, but interesting if they can get the kinks worked out.