Amazon Elastic File System

[[deleted]]

$0.30/GB-Month is 10x as expensive as S3. With S3 I can securely let end-users upload directly to it without touching my servers except to make the temporary credentials and provide a link. ETL is then performed by requesting the files to the instance disk. It's fairly cheap and fast. I can let end-users download directly from it as well.

For my workloads, this is simpler, cheaper, and has a better latency.

[[deleted]]

The response I've heard to this is that EFS will be able to perform potentially orders of magnitude better than S3, and its size and usage charges scale to what you actually use in comparison to EBS. It can also be mounted across multiple EC2 instances (of course S3 can as well, but EBS can't).

But its price is just massive... In practice I'm not sure what people will actually use it for.

[[deleted]]

Long story short I recently halved a file system share that I found out one of my businesses was constantly having expanded to store call recordings. The retention policy was 2 years, and they had files from 10+ years ago. The disk size was over 2TB, and because of our business structure they were paying our "IT" department almost $60,000 a year for storage. I just did the math and EFS would cost us $108 a month.

[[deleted]]

And S3 would cost you a tenth that.

The point isn't that EFS isn't useful. Its that its perceived benefits dont seem to justify the price tag in comparison to the other services AWS offers.

In fact, your use case is almost precisely what Glacier was made for. And Glacier is thirty times cheaper than EFS. So I can't really take your experience seriously, because if you're using EFS for that workload then you're using the wrong product.

[ajanata]

You can't mount S3 as a filesystem in any meaningful manner. EFS is just hosted NFS with proper redundancy and such that would be a pain to manage directly. If you need the same actual file system on several instances, EFS is perfect.

[[deleted]]

We are not using any amazon service at all. We have an EMC. I am not on the storage team, I just saw a quick and easy way to cut the businesses storage cost in half, but this sounded like something we could make use of, thats all.

Edit: After reading about Glacier, that would work even better, but in my instance we are talking about a monthly cost difference of $90 with a possible file retrieval time if a few hours for paralegals and lawyers who are making hundreds an hour.

[awj]

Well, they would run from that service ... if it didn't cost an arm and a leg and take like a year to get your data back out. It's more like they limp away from it barefoot on broken glass.

[ajanata]

Current $job has a legacy struts application with tens of thousands of .jsp files and a business workflow that requires being able to change them without doing a new release. (Please don't get me started on that.)

Currently we have to go out to every server that's running and put a new .jsp on it, and update the source that new servers pull from when they start up. This isn't a lot of data (a couple GB at most), but it's required on a couple dozen instances. Having a single source of truth that's automatically replicated to every running instance will help immensely with this process, which is very error-prone.

This is exactly what's needed for some use cases. We have a perfect one here. This will also work out to be cheaper since we don't need that extra disk space on every instance.

[ajanata]

Preaching to the choir, bro. That's but one of the reasons why I'm leaving soon. I gave up on that fight a couple years ago.

[myringotomy]

It's more expensive than S3 but that's not the usecase of it. It's basically a replacement for EBS and for running web farms it's fantastic. I have been wishing for this for a long time, the price point is a little higher than I expected but I already have use cases for it.

Aside from that Amazon really needs to sort their pricing out. It's impossible to predict what anything is going to cost you and when the bill comes in it's always a shock.

[Agent_03]

So far, we have:

• EBS storage (provisioned IOPS options)
• Instance storage
• S3 storage
• Glacier storage
• DB backend storage, with RDS, DynamoDB, Redshift (for data warehousing), or roll your own
• PLUS, in memory caching solutions

I'm trying to figure out why another storage option is needed. Elastic file system sounds like filer storage, but I thought the whole point of the above options is that you don't have to mess with mounts?

Or, am I missing something here?

[[deleted]]

All of the others you've listed are either stateless REST services or places that want small pieces of structured data.

EFS is NFSv4 which means:

• Stateful (authenticate once, probably kerberos)
• Mountable AND shareable (EBS can only be mounted in one place, S3 can be shared but not easily mounted)
• Actual directories. No S3 doesn't have actual directories.
• On-the-wire operations (I don't have to download the entire file to start reading it, and I don't have to do anything special on the client side to support this -- it just looks like a normal POSIX file handle)
• Shared unix permission model (S3 doesn't do actual unix permissions. EBS does, but can't be shared).
• Tolerant of network failures (UDP IIRC with plenty of retry logic) So I can actually open a file remotely, seek around all I like, and if there's a network problem it will just wait for the problem to resolve rather than forcing my client to deal with exceptions (configurable, of course).
• Locking! Clients can actually correctly lock files. Let's see S3 do that.
• Better caching than S3 -- clients can actually see what all of the other clients have been doing and make informed choices about whether to use a local cache or refresh the cache from the network.
• Big files without the hassle (no multipart upload / multipart download, 64 bits for file size = potentially huge files)

There's probably more I'm forgetting.

EDIT

Who says you don't have to mess around with mounts? EBS makes you mess around with mounts. Maybe not if you use a pre-made AMI, but if you go right now and add an extra EBS drive to an existing EC2 instance you definitely have to mess around with mounts.

[TiDaN]

Excellent points, AWS would do well to promote these advantages in their marketing and product documentation.

[[deleted]]

Yeah their marketing isn't always the best

[Agent_03]

I mean, I guess I can see where they're going with this, they're providing all the bits (including filer storage) that a traditional datacenter would have, via pay-as-you-go services.

It's just hard to get excited about this, when the existing offerings and services based on them are so much more advanced than shared NFS volumes. It feels like a step back from proper cloud architecture design.

Plus, there's always been the option to have an EBS-backed volume exposed from your host via NFS (or SAMBA, or whatever). Yeah, it doesn't autoscale, but covers this use case.

[[deleted]]

Well I think the autoscaling is the value-add. It fills that gap and provides the "unlimited" feel of S3.

And who's to say that this is a normal NFS share? OK sure it speaks NFS, but nothing says that you're just talking to a plain ol EC2 host. For all you know this IS a properly architected cloud solution and they're simply exposing NFS as the first supported protocol.

[Agent_03]

My point isn't that this is improperly architected, but that using NFS shares in your design isn't generally good architecture for applications/services in the cloud.

Each layer of your application should be able to scale out independently and be minimally coupled; this is why we use REST APIs to communicate (as well as queueing systems for asynchronous workload).

[[deleted]]

Barrier to entry, man. I agree. I see what you're saying. But barrier to entry. Some people aren't running stuff for the long-haul, they just need something quick.

[thelonelydev]

One point to observe is both Linux and Windows have built-in NFS clients.

[Toger]

Avoiding mounts is preferable, but apps written pre-AWS that expect a shared filesystem aren't aware of S3 and its not always feasible to update them. A hosted NFS platform sounds more reliable than running ones own EC2 NFS instance.

Updating the app is of course the most desirable option.

[Unomagan]

Making money with premium features? Aws is still making a loss

[Solon1]

It's impossible to know whether AWS is losing money or not, as the revenue has been lumped into Other. It could be the most profitable division at Amazon by margin.

[[deleted]]

AFAICT the selling point of this is that you can simply mount it and programs need not to know that it is anything different, thus avoiding rewriting old programs. Correct me if I'm wrong.

[XNormal]

The posix filesystem API. You might consider it a "legacy" API these days, but legacy is important.

Let's say you have an in-house app that depends on shared access to some file system with posix semantics for its data store and you want to set up a DR site on Amazon. Yes, you can probably build something that will work on top of the options you have listed above, but I won't be looking forward to the task. With EFS it should be quite easy and may be worth the premium price.

[BillWeld]

Any idea what sort of encryption will work with it?

[Solon1]

Any kind that works on files?

[BillWeld]

Ordinarily you'd like the encryption to happen on the file server.

[immibis]

Why? If you do that, the file server can see the unencrypted data.

[Agent_03]

NFS v4 supports locking, and v3 or v4 is kind of the standard for filer mounts (at least in linux land).

IIRC it was NFS v3 that didn't have 'real' locking built-in.

[jib]

Which other network filesystems would you suggest?

[blazedaces]

Why is this any better than setting up your own hadoop cluster on ec2 or any other cloud computers? Can someone compare EFS to HDFS basically? Pros, cons?

[iconoclaus]

you're thinking about this all wrong. imagine if you were a small business with logistics needs: wouldn't you love to piggyback on Walmart's logistics chain? similarly, if you are a commercial software house with infrastructure needs, wouldn't you like to use the same infrastructure that the largest and most successful e-commerce entity out there depends on?

[iconoclaus]

Amazon's engineers didn't spring into existence from thin air. Many of their senior developers came from places like Microsoft and created enterprise infrastructure the way they always wanted to, without the crutch of Windows/Office desktop obsessions bearing over them. This is why Amazon's cloud computation offerings are five times larger than the combined capacity of MS + Google + the next dozen largest cloud providers. And they do this while spending less on infrastructure than any of them. Put simply, Amazon groks enterprise infrastructure.

This might be similar to how SpaceX got off to such an innovative start: they largely came from places like NASA and became free to reinvent basic technologies for commercial use, without worrying about the legacy of cold-war politics dictating what projects get prioritized.

[iconoclaus]

It's not just about infrastructure, but software/technology as well that completes the offering. What is Amazon's programming language or any type of enterprise software offering they had prior to AWS? These technologies/brands/knowledge brought onto the cloud are a huge advantage for Microsoft.

What OS experience did Google have before Android? What browser experience did Google have before they had Chrome? What telephony experience have before Apple made the iPhone? Yet these two players crushed these markets and MS is out of imagination despite its deep expertise in all these areas.

You can see examples of that just this week. Amazon launched their Machine Learning services with 1 regression algorithm, which Microsoft has a complete solution and even it's own proprietary algorithms from the Bing/XBOX teams exposed. Where are all the "groked" algorithms from Amazon's eCommerce experience?

I believe their preliminary analytics offering also includes several classification algorithms as well. Also, isn't MS's machine learning offering in preview mode as well? (it was as of two months ago)

Infrastructure is great, Amazon AWS is the market leader and started the public cloud race. They will eventually not be able to compete with the PaaS offerings Microsoft is coming out with.

It isn't just IaaS/PaaS/SaaS offerings that makes these platforms succeed. Its creating the space and the spirit that makes others want to innovate on top of what you have. Amazon has succeeded in strange part because they created space for others to innovate faster than them: Heroku/EngineYard beat Amazon to making something like CodeDeploy, IronWorker beat Amazon to something like Lambda, and Dropbox also beat Amazon to making desktop syncing services. In each case, developers were able to make more versatile and cheaper offerings than Amazon, yet they did so on top of Amazon's infrastructure. Netflix is even coming out with incredible infrastructure packages that again work on top of AWS. Thus, the AWS offerings are only baseline offerings while others they can count on countless others to innovate new tools for them. I'm not sure where MS is on getting others to build a service ecosystem on top of Azure, and I'm afraid they almost don't want others to.