[amazon-web-services] AWS EFS vs EBS vs S3 (differences & when to use?)

As per the title of this question, what are the practical differences between AWS EFS, EBS and S3?

My understanding of each:

• S3 is a storage facility accessible any where
• EBS is a device you can mount onto EC2
• EFS is a file system you can mount onto EC2

So why would I use EBS over EFS? Seem like they have the same use cases but minor semantic differences? Although EFS is replicated across AZs where as EBS is just a mounted device. I guess my understanding of EBS is lacking hence I'm unable to distinguish.

Why choose S3 over EFS? They both store files, scale and are replicated. I guess with S3 you have to use the SDK where as with EFS being a file system you can you standard I/O methods from your programming language of choice to create files. But is that the only real difference?

In simple words

Amazon EBS provides block level storage .

Amazon EFS provides network-attached shared file storage.

Amazon S3 provides object storage .

Fixing the comparison:

• S3 is a storage facility accessible any where
• EBS is a device you can mount onto EC2
• EFS is a file system you can mount onto several EC2 instances at the same time

At this point it's a little premature to compare EFS and EBS- the performance of EFS isn't known, nor is its reliability known.

Why would you use S3?

• You don't have a need for the files to be 'local' to one or more EC2 instances.
• (effectively) infinite capacity
• built-in web serving, authentication

Apart from price and features, the throughput also varies greatly (as mentioned by user1677120):

EBS

Taken from EBS docs:

| EBS volume | Throughput |           Throughput          |
|    type    |   MiB/s    |         dependent on..        |
|------------|------------|-------------------------------|
| gp2 (SSD)  | 128-160    | volume size                   |
| io1 (SSD)  | 0.25-500   | IOPS (256Kib/s per IOPS)      |
| st1 (HDD)  | 20-500     | volume size (40Mib/s per TiB) |
| sc1 (HDD)  | 6-250      | volume size (12Mib/s per TiB) |

Note, that for io1, st1 and sc1 you can burst throughput traffic to at least 125Mib/s, but to 500Mib/s, depending on volume size.

You can further increase throughput by e.g. deploying EBS volumes as RAID0

EFS

Taken from EFS docs

| Filesystem |    Base    |   Burst    |
|    Size    | Throughput | Throughput |
|    GiB     |   MiB/s    |   MiB/s    |
|------------|------------|------------|
|         10 |        0.5 |        100 |
|        256 |       12.5 |        100 |
|        512 |       25.0 |        100 |
|       1024 |       50.0 |        100 |
|       1536 |       75.0 |        150 |
|       2048 |      100.0 |        200 |
|       3072 |      150.0 |        300 |
|       4096 |      200.0 |        400 |

The base throughput is guaranteed, burst throughput uses up credits you gathered while being below the base throughput (so you'll only have this for a limited time, see here for more details.

S3

S3 is a total different thing, so it cannot really be compared to EBS and EFS. Plus: There are no published throughput metrics for S3. You can improve throughput by downloading in parallel (I somewhere read AWS states you would have basically unlimited throughput this way), or adding CloudFront to the mix

AWS EFS, EBS and S3. From Functional Standpoint, here is the difference

EFS:

1. Network filesystem :can be shared across several Servers; even between regions. The same is not available for EBS case. This can be used esp for storing the ETL programs without the risk of security

2. Highly available, scalable service.

3. Running any application that has a high workload, requires scalable storage, and must produce output quickly.

4. It can provide higher throughput. It match sudden file system growth, even for workloads up to 500,000 IOPS or 10 GB per second.

5. Lift-and-shift application support: EFS is elastic, available, and scalable, and enables you to move enterprise applications easily and quickly without needing to re-architect them.

6. Analytics for big data: It has the ability to run big data applications, which demand significant node throughput, low-latency file access, and read-after-write operations.

EBS:

1. for NoSQL databases, EBS offers NoSQL databases the low-latency performance and dependability they need for peak performance.

S3:

Robust performance, scalability, and availability: Amazon S3 scales storage resources free from resource procurement cycles or investments upfront.

2)Data lake and big data analytics: Create a data lake to hold raw data in its native format, then using machine learning tools, analytics to draw insights.

3. Backup and restoration: Secure, robust backup and restoration solutions
4. Data archiving
5. S3 is an object store good at storing vast numbers of backups or user files. Unlike EBS or EFS, S3 is not limited to EC2. Files stored within an S3 bucket can be accessed programmatically or directly from services such as AWS CloudFront. Many websites use it to hold their content and media files, which may be served efficiently via AWS CloudFront.

This question is very much answered by other people, I just want to make a point whenever deciding on any service to be in AWS is that understanding the use case for each and also see the solution that the service will provide in terms of the Well-Architected Framework, do you need High Availability, Fault Torelant, Cost optimization. This will help to decide on any kind of service to be used.

The main difference between EBS and EFS is that EBS is only accessible from a single EC2 instance in your particular AWS region, while EFS allows you to mount the file system across multiple regions and instances.

Finally, Amazon S3 is an object store good at storing vast numbers of backups or user files.

I wonder why people are not highlighting the MOST compelling reason in favor of EFS. EFS can be mounted on more than one EC2 instance at the same time, enabling access to files on EFS at the same time.

(Edit 2020 May, EBS supports mounting to multiple EC2 at same time now as well, see: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volumes-multi.html)

EFS & S3 have the same purpose, you can store any kind of object or files.

But for me the only difference is EFS is allowing you to have a traditional file system in the VM(EC2) cloud with more flexibility like you can attach to multiple instances.

S3, on the other hand, is a separate flexible and elastic server for your objects. It can be used for your static files, images, videos or even hosting static app (js).

EBS is obviously for block storage where you can install OS or anything related to your OS.

EBS is simple - block level storage which can be attached to an instance from same AZ, and can survive irrespective of instance life.

However, interesting difference is between EFS and S3, and to identify proper use cases for it.

Cost: EFS is approximately 10 times costly than S3.

Usecases:

• Whenever we have thousands of instances who needs to process file simultaneously EFS is recommended over S3.
• Also note that S3 is object based storage while EFS is file based it implies that whenever we have requirement that files are updated continuously (refreshed) we should use EFS.
• S3 is eventually consistent while EFS is strong consistent. In case you can't afford eventual consistency, you should use EFS

Amazon EBS provides block level storage - It is used to create a filesystem on it and store files. Amazon EFS - its shared storage system similar like NAS/SAN. You need to mount it to unix server and use it. Amazon S3 - It is object based storage where each item is stored with a http URL.

One of the difference is - EBS can be attached to 1 instance at a time and EFS can be attached to multiple instances that why shared storage. S2 plain object storage cannot be mounted.

To add to the comparison: (burst)read/write-performance on EFS depends on gathered credits. Gathering of credits depends on the amount of data you store on it. More date -> more credits. That means that when you only need a few GB of storage which is read or written often you will run out of credits very soon and througphput drops to about 50kb/s. The only way to fix this (in my case) was to add large dummy files to increase the rate credits are earned. However more storage -> more cost.