Amazon Now Serves Files At Sub-Millisecond Speeds

segaboy81 writes: Amazon is announcing an enormous increase in read speeds for its new and existing Elastic file systems. According to an AWS blog post, EFS read operations have typically hovered in the low 1ms range, but after they "flipped the switch," read operations are now halved. Users can now expect read speeds as low as 600 micro-seconds.

Which is excellent

[pele]

Now for those sub-millisecond pings and jitters and pcie-speed internet and we're all set!

Re:

[timeOday]

I think the point is a single remote request made to a cloud service may require dozens of filesystem reads within that cloud to process.

Re:

[Neuroelectronic]

It may but it doesn’t. However there may be several disk writes for logging, which is nonblocking. Everything else is cached, when it comes to access restriction and so on.

Likely the only upgrade here is a larger cache in front if EBS, serving something like 50-90% of requests.

Re:

[r1348]

AWS target is not the end user, but rather those companies who run their large scale CDNs over AWS.
For them, a reduction of file system latency scales up a lot.

now...

[swan5566]

..if they would only reduce the cost of EFS so that it's a viable option for my project...

Re:

[kriston]

Came here to say this. It's cheaper to run your own EC2 instance with NFS than it is to run EFS.

Also, you don't get full-speed EFS until your filesystem reaches past a size threshold. Last I checked it was 1 TB which makes high-IOPS EFS prohibitively expensive for most applications.

Re:

[bill_mcgonigle]

Most of AWS is for people who can't run their own infrastructure and can't afford to hire that out. AWS falls right in the middle, with the added bonus of someone outside the company to blame.

That's a value-add for most IT managers

Yawn

[Viol8]

This is supposed to be a technical site so how about some technical info? Have they switched en mass to SSDs, improved their routing, file system, network infrastructure? What? Otherwise this is just a puff piece to make Bezos a bit richer.

Re:

[DrMrLordX]

It's just an ad for AWS.

Re:

[dogsbreath]

I agree

Does /. get a kickback for binspam?

Re:

[DrMrLordX]

Probably.

Re:

[nospam007]

"This is supposed to be a technical site so how about some technical info? Have they switched en mass to SSDs, improved their routing, file system, network infrastructure? What? "

They pressed the Turbo button.

Re:

[seth_hartbecke]

I think the bigger point is: Nobody outside of AWS knows.

This is one of the "dangers" of the cloud. Increasingly larger and larger portions of our server infrastructure is now a trade secret. A very small and shrinking of people have any idea how any of the server infrastructure actually works.

Re:Yawn

[Tailhook]

how about some technical info

I agree with DrMrLordX; this is just an ad for AWS.

However, the new performance feature is real. So what they're talking about here is a distributed Network File System (NFS) service managed by AWS called Elastic File System (EFS.) It provides high durability NFS service within an AWS Region.

EFS is known for being expensive and slow; as others have pointed out you can run your own NFS server on an EC2 instance and get better performance for less money. However, that approach won't survive the failure of the "Availability Zone" running the EC2 instance (each "Region" is composed of multiple AZs.) EFS is supposed to survive AZ failure so, in theory, your EFS file system will keep working.

AWS has been incrementally improving EFS performance for years; this is only the latest improvement delivered since EFS was launched. The reduced latency is achieved using a distributed cache over EFS. The cache is transparent and consistent, which is a tough problem to solve in distributed systems, so this is a significant technical achievement at no additional cost.

Even with the latency reduction EFS is still a relatively low performance file system. If you need it it's an important capability, but the poor performance and high cost limits viable use cases.

My own view is that EFS solves a problem almost no one cares about. In theory you use Region wide services to survive AZ failure, but in practice when big things fail at AWS it takes out a whole Region* so High Availability means you must distribute across Regions, and EFS is local to a Region.

Also, EFS doesn't provide snapshots, so making crash consistent backups becomes a can of worms and this further limits use cases.

*us-east-1, for example, goes down about annually

Re:

[afidel]

Hmm, distributed cache with sub-ms latencies, I wonder if they borrowed any code from FSX for Lustre since that sounds a lot like Lustre to me =)

Re:

[hebcal]

The real story is how bad latency used to be. Basically it used to be easy to find yourself with 1-2 SECOND read latency in EFS. Hopefully with this and other updates these problems are now firmly in the past. https://forums.aws.amazon.com/... [amazon.com] https://forums.aws.amazon.com/... [amazon.com]

Simspns did it

[jacks smirking reven]

Benjamin: I invented a program that downloads porn off the internet one million times faster.
Marge: Does anybody need that much porno?
Homer: (drooling) Oh, one million times...

my computer is faster then that

[maybe111]

even if I dont account for the network lag that amazon is a victim of...

Oh Goody

[ElizabethGreene]

This is super exciting. Now pages can load 15MB of JavaScript even FASTER! YEAH! :(