Containers
Unlocking AWS Fargate feature for attaching Amazon EBS Volumes to ECS Tasks
Introduction
Amazon Elastic Container Service (Amazon ECS) has now enhanced its functionalities by integrating support for Amazon Elastic Block Store (Amazon EBS) volume attachment to Amazon ECS tasks. This feature simplifies using Amazon ECS and AWS Fargate with Amazon EBS. Amazon ECS facilitates seamless provisioning and attachment of EBS volumes to ECS tasks on both Fargate and Amazon Elastic Compute Cloud(Amazon EC2) platforms. In Amazon ECS tasks, you have the flexibility to select EBS volume attributes, such as size, type, IOPS, and throughput, tailoring the storage to meet the specific needs of your application. Additionally, the capability to create volumes from snapshots allows for the rapid deployment of new tasks with pre-populated data.
With this feature, Amazon ECS streamlines the deployment of storage-heavy and data-intensive applications, such as ETL processes, media transcoding, and machine learning (ML) inference. For a comprehensive understanding of integrating Amazon ECS with Amazon EBS, see Channy Yun’s launch post, which offers detailed guidance on getting started with this integration.
In this post, we discuss performance benchmarking results for Fargate tasks using EBS volumes. The goal aims to assess the performance profiles of various EBS volume configurations under simulated workloads. The insights garnered from this analysis can assist you in identifying the optimal storage configurations for I/O intensive workloads.
For context, the data and observations presented in this post are specific to the Oregon Region, reflecting the state of the Fargate’s On-Demand data plane as observed in February 2023. Note that the situation might have changed, offering a different landscape today.
EBS volume types
Amazon EBS offers a range of block storage volumes, leveraging both Solid State Drive (SSD) and Hard Disk Drive (HDD) technologies to cater to different workload requirements:
- General Purpose SSD volumes (gp2 and gp3)
- Provisioned IOPS SSD volumes (io1 and io2 Block Express)
- Throughput Optimized HDD volumes (st1)
- Cold HDD volumes (sc1)
General Purpose SSD volumes are the most commonly used block storage volume. Backed by solid-state drives, these volumes offer a balanced performance for a broad range of transactional workloads, including boot volumes, medium-sized databases, and low-latency interactive applications. They strike an optimal balance between cost and performance, making them suitable for a variety of use cases that demand consistent, moderate IOPS with reliable throughput.
Provisioned IOPS SSD io1 and io2 volumes feature solid-state drives, marking them as Amazon EBS’s storage solutions for high IOPS and low latency needs. Both are tailored for critical applications that demand consistent, rapid access, promising IOPS reliability at a 99.9% rate, suitable for high-performance databases and applications. However, io2 differentiates itself by offering increased durability, larger capacity options, and consistent latency. However, both volumes serve distinct needs depending on the specific requirements of the workload, making sure of flexibility in choice.
Throughput Optimized HDD st1 volumes are designed to offer low-cost magnetic storage prioritized for throughput over IOPS. These volumes align with the needs of workloads that benefit from large, sequential reads and writes, making them ideal for processes such as big data analytics, log processing, and data warehousing.
Cold HDD sc1 volumes, similar to st1 volumes, focus on throughput but at a more economical rate and with a lower threshold. Best suited for less frequently accessed, sequential cold data, these volumes represent a lowest-cost solution for storage needs that don’t demand constant access.
Testing methodology
We tested each EBS volume type across multiple Fargate task sizes with XFS. The baseline EBS volume IOPS and throughput available for a Fargate task depend on the total CPU units you request. The difference in storage performance is clear in the results. For example, tasks with 16 vCPUs provide higher IOPS and throughput as compared to tasks with 0.25 vCPUs.
To make sure of a thorough examination, we explored a spectrum of Fargate task sizes, ranging from tasks allocated with 0.25 vCPUs up to those with 16 vCPUs, across the following configurations:
- .25 vCPU | 1 GB
- .5 vCPU | 2 GB
- 1 vCPU | 4 GB
- 2 vCPU | 6 GB
- 4 vCPU | 8 GB
- 8 vCPU | 16 GB
- 16 vCPU | 32 GB
Our testing methodology for General Purpose SSD and Provisioned IOPS SSD volumes involved conducting 16 KB random read and write operations, adhering to the guidelines specified in the EBS documentation. For tasks equipped with Throughput Optimized HDD or Cold HDD volumes, our approach entailed executing 1 MiB sequential read and write operations to better gauge their performance under workload conditions typical for these storage types.
By repeating each test three times and calculating the mean values, we aimed to make sure of the reliability and accuracy of our performance measurements.
General purpose SSD – gp3 volumes
Given the versatility and price-to-performance ratio of gp3, we expect this volume type to be the most commonly used block storage for Fargate tasks. gp3 volumes deliver a baseline performance of 3,000 IOPS and 125 MiB/s at any volume size. Fargate supports gp3 volumes with support for a maximum of 16,000 IOPS and 1,000 MiB throughput.
We performed tests on gp3 volumes configured with 2,000 GiB size, 16,000 IOPS, and 1000 MiB/s throughput to guarantee maximum storage performance.
We learned Fargate offers consistent IOPS performance across most task sizes. Tasks with 0.25 vCPU and 1 GB memory are an outlier in this group, as they do not deliver the maximum 16,000 IOPS. Task sizes from 1 vCPU onward achieve the maximum configured IOPS.
Tasks with 0.25 vCPU couldn’t go beyond 200 MiB/s and 150 MiB/s in read and write tests respectively.
General Purpose SSD – gp2 volumes
We recommend customers opt for gp3 volumes over gp2 for several reasons. Firstly, gp3 volumes allow for the provisioning of IOPS independently from storage capacity, offering more flexibility. Secondly, they are more cost-effective, with a 20% lower price per GB compared to gp2 volumes.
The gp2 volume performance relies on a burst bucket model, where the size of the volume dictates its baseline IOPS. This baseline determines the rate at which the volume accumulates throughput credits. For those customers with specific needs, Fargate continues to support gp2 volumes.
Our decision to include gp2 volumes in our benchmarking was straightforward, as our testing setup was already compatible. We benchmarked gp2 volumes at a size of 6,000 GiB. At this size, gp2 volumes can achieve 16,000 IOPS, the maximum for gp2, due to the volume size proportionally influencing the IOPS allocation.
The IOPS performance on gp2 volumes was consistent across all task sizes except 0.5 and 0.25 vCPUs. Tasks with 1 vCPU and larger achieved maximum provisioned throughput of 16,000 IOPS.
Throughput performance on gp2 was very similar to gp3 volumes. The test results offer another proof of why customers should prefer gp3 over gp2.
Provisioned IOPS SSD – io1 volumes
Amazon EBS io1 volumes are ideal for both IOPS-intensive and throughput-intensive workloads that require low latency and have moderate durability requirements or include built-in application redundancy. io1 and io2 volume types provide higher throughput and IOPS compared to gp3 volumes.
We performed tests on io1 volumes configured with 2,000 GiB size and 64,000 IOPS
Only tasks with 8 or more vCPUs achieved more than 20,000 IOPS. Even though the io1 volumes attached to the tasks supported up to 64,000 IOPS, none of the tasks approached the maximum IOPS mark in our tests. Considering these results, gp3 may turn out to be more cost-effective storage for tasks with fewer than 8 vCPUs.
Tasks with io1 volumes reported more I/O throughput as compared with gp3. For applications that need higher throughput and IOPS, io1 volumes are more suitable. All tasks, except 0.25 vCPU, achieved at least 300 MiB/s of throughput. Compare this to gp3, which achieved a maximum of 260 MiB/s.
Provisioned IOPS SSD – io2 Block Express volumes
Amazon EBS io2 Block Express offers the highest performance block storage in the cloud with 4x higher throughput, IOPS, and capacity than gp3 volumes, along with sub-millisecond latency. io2 Block Express is designed to provide 4,000 MB/s throughput per volume, 256,000 IOPS/volume, up to 64 TiB storage capacity, and 1,000 IOPS per GB as well as 99.999% durability.
The io2 volumes we used in benchmarking had 2,000 GiB size and 64,000 IOPS.
io2 volumes attained more IOPS on Fargate than io1 volumes on tasks with more vCPUs. However, the IOPS performance of io1 and io2 volumes is identical for tasks with less than 8 vCPUs. Even tasks with 8 and 16 vCPUs achieved about 40,000 IOPS on io2 volumes with 64,000 provisioned IOPS.
Note that random write performance on tasks with io2 volumes was much higher than io1, but it’s only applicable for larger tasks.
The throughput scaling with task size observed with io2 volumes is similar to that of io1, with io2 volumes demonstrating higher write throughput. Note that in most scenarios, io2 is a more advantageous choice over io1. Although both volume types start at the same price point, io2’s tiered IOPS pricing model makes it a more cost-effective option for configurations requiring high IOPS.
Throughput Optimized HDD – st1
Throughput Optimized HDD (st1) volumes provide low-cost magnetic storage that defines performance in terms of throughput rather than IOPS. This volume type is a good fit for large, sequential workloads, such as Amazon EMR, ETL, data warehouses, and log processing. st1 volumes offer low-cost storage for workloads that need large and sequential I/O.
Like gp2, st1 uses a burst bucket model for performance. Volume size determines the baseline throughput of your volume, which is the rate at which the volume accumulates throughput credits. st1 volumes provide burst throughput of up to 500 MiB/s.
We configured st1 volumes with 13,000 GiB size, which results in a base throughput of 500 MiB/s.
st1 volumes are throughput optimized, throughput is a more appropriate measurement of performance. We’ve included IOPS results to be consistent. To summarize, all tasks with over 1 vCPU attain about 500 IOPS.
st1 offers consistent throughput across most task sizes. Although io1 and io2 provide over 500 MiB/s throughput on tasks with 8 and 16 vCPUs, st1 offers about 500 MiB/s throughput on most task sizes. This makes st1 better suited for workloads that need higher throughput with smaller task sizes.
Cold HDD – sc1 volumes
Cold HDD (sc1) volumes provide low-cost magnetic storage that, like st1, defines performance in terms of throughput rather than IOPS. sc1 volumes have lower throughput than st1, making them ideal for large, sequential cold-data workloads. sc1 (such as gp2 and st1) also uses a burst bucket model. Volume size determines the baseline throughput.
We maxed the size of sc1 volumes to 16 TiB storage in order to guarantee the maximum throughput of 192 MiB/s. All sc1 volumes have burst throughput of 250 MiB/s.
Our tests showed that sc1 volumes achieved about half of the IOPS when compared to st1 volumes. Once again, tasks with one and more vCPUs had consistent IOPS performance.
sc1 volumes reported about half of the throughput when compared to st1 volumes. Given that sc1 volumes cost a third of a similarly sized st1 volume, sc1 volumes are great for workloads that need infrequent access to data.
Conclusion
This post reviewed Amazon EBS performance across different Fargate task sizes. It found that for the majority of workloads on Fargate, gp3 volumes, aptly named for their general-purpose use, are appropriate. However, it is advised against using io1 and io2 volumes for tasks requesting 0.25, 0.5, or 1 vCPU due to insufficient CPU cycles to leverage their potential for more than 30,000 IOPS and 300 MiB throughput. Instead, these high-performance volumes are recommended for workloads requiring significant IOPS and throughput. For tasks needing sequential I/O, st1 volumes, or the more economical sc1 volumes, are also beneficial.