AWS Storage Blog
Achieve higher database performance using Amazon EBS io2 Block Express volumes
I think it’s safe to say that every organization that runs its own IT environment has applications running on a database. Managing database servers is a critical job, as any performance hiccups on the database could affect many users and the workloads that they execute. Many enterprises see negative impact on the performance of their applications due to the latencies observed in storage when running their database workloads. To sustain I/O intensive workloads and provide superior performance for databases, AWS announced Amazon EBS io2 Block Express in December of 2020, which simplifies the storage required for high-performance database servers.
In this blog post, I discuss how the performance compares for an Oracle database running on Amazon EBS gp3 vs io2 Block Express. Amazon EBS io2 Block Express provides advantages of some advanced communication protocols implemented as part of the AWS Nitro System. You can achieve up to 256,000 IOPS and 4000 MB/s per volume of throughput with a maximum volume size of 64 TiB, all with low-variance sub-millisecond disk I/O latency.
Database performance comparison overview
When running a database workload, the main performance metrics to consider for database storage are IOPS, latency, and throughput.
- IOPS measures number of input and output operations per second.
- Latency is the time it takes a single I/O request to complete.
- Throughput is the amount of data that can be read or written per second.
In order to measure database performance, I used Swingbench, which is a load generator and benchmarking tool designed to stress test Oracle databases. Stress test benchmarks for swingbench provide us information about transactions per minute executed on the databases. Using the transaction per minute metric for the same stress test, comparison was made for performance of Amazon EBS gp3 vs io2 Block Express.
I provisioned two environments on Amazon EC2 using the r5b.24xlarge instance type to examine the impact of storage performance on databases. One environment was configured with Amazon EBS gp3 volume type and the other with Amazon EBS io2 Block Express volume type. I used three types of tests, load, stress, and latency, by configuring Oracle databases on each of these environments.
Oracle Automatic Storage Management (ASM) is a recommended solution to use when migrating Oracle databases onto Amazon EC2 instances with EBS volumes. Hence, in this blog, benchmarks are executed using Oracle ASM configured on gp3 and io2 Block Express to compare performance.
I compare the database transactional and latency performance using the different storage volume types of io2 Block Express and gp3, with the same IOPS and throughput configuration on each. Based on the analysis of results, io2 Block Express performed 52% higher on IOPs and provided 87% lower latency compared to gp3 volumes, at 74.57% higher cost. Hence, the io2 Block Express is a recommended solution for database workloads that are sensitive to disk latencies and where performance is a priority over cost. Detailed test results are shown in the following sections.
Comparing Amazon EBS volume types gp3 and io2 Block Express
Here is a quick comparison of features and pricing between gp3 and io2 Block Express volumes.
| Amazon EBS volume type | gp3 | io2 Block Express |
| Durability | 99.8% – 99.9% durability (0.1% – 0.2% annual failure rate) | 99.999% durability (0.001% annual failure rate) |
| Use cases |
|
|
| Volume size | 1 GiB – 16 TiB | 4 GiB – 64 TiB |
| Max IOPS per volume (16 KiB I/O) | 16,000 | 256,000 |
| Max throughput per volume | 1,000 MiB/s | 4,000 MiB/s |
| Max IOPS/Instance | 260,000 | 260,000 |
| Max Throughput/Instance | 7,500 MiB/s | 7,500 MiB/s |
| Latency | Single digit millisecond | Sub-millisecond |
| Price | Storage price:
IOPS price:
Throughput price:
|
Storage price:
IOPS price:
|
Benchmarking system configuration
I used the following configuration to run performance tests for the comparison:
- An Amazon EC2 instance of type r5b.24xlarge with 96vCPU/768GB RAM.
- Operating system: Red Hat Enterprise Linux Server release 7.9.
- Oracle database: Version 12.1.0.2.
- Storage configuration: I provisioned the storage for gp3 and io2 Block Express to match IOPS (80000 IOPS) and throughput (8000 MBps) for both configurations. This resulted in eight gp3 volumes (per volume: 10000 IOPS, 1000 MBps) versus two io2 Block Express volumes (per volume: 40000 IOPS, 4000 MBps) to achieve the same total performance metrics.
- Storage comparison configuration: I compared performance using Oracle ASM for both gp3 and io2 Block Express.
| Configuration | Oracle ASM | |
| gp3 | io2 Block Express | |
| Storage configuration for data and logs | 8 x 1 TB, allocation unit 1 MB | 2 x 2 TB, allocation unit 1 MB |
| Operating system | Red Hat Enterprise Linux 7.9 | |
| EC2 instance type | r5b.24xlarge | |
| Region | us-east-1 | |
| Availability Zone | us-east-1a | |
| Memory | 768 GB | |
| CPU | 96 vCPU | |
| # Database volumes | 8 | 2 |
| Max IOPS | 10,000/volume | 40,000/volume |
| Total IOPS | 80,000 | |
| Max throughput | 1000 MBps/volume | 4000 MBps/volume |
| Total throughput | 8000 MBps | |
Benchmarking data generation
To compare the performance of Amazon EBS gp3 and io2 Block Express, I loaded my Oracle database with around 500 GB of data using the sbutil tool. This tool generates an online transaction processing (OLTP) schema.
For more information on how the data was created for benchmarking purposes, refer to this link.
Benchmark performance analysis
To benchmark performance, I generated the load using the swingbench tool, and stress tests were executed for each of the different storage configurations for 30-minute durations. Each stress test simulated a workload for 150 users.
Benchmark test overview
The following three tests were performed against Oracle database with ASM configured on gp3 and io2 Block Express to compare performance.
- Load test: I used this article as a reference and kept the same memory configuration. Then, I tested databases with different storage configurations, using both io2 Block Express and gp3 and gradually increased the schema size to determine transactions per minute for congruent loads.
- Stress test: This test simulates the load of 150 concurrent sessions for 30 minutes on the databases. For the same amount of load, Oracle AWR report in all three configurations to compare against the number of transactions per second was generated.
-
- This test gives us a result of impact on transactions per second with high user concurrency when using different storage configurations.
- Latency test: In this test, while executing the same stress test on the databases with different storage configurations, gp3 and io2 Block Express, I measured the Amazon EBS latency. This test shows the impact on disk latency when maximum IOPS was reached on different storage configurations.
Benchmark test results
In this section, I give details about the results from the benchmark tests executed against Oracle database running with Oracle ASM on Amazon EBS io2 Block Express and gp3.
Load test
Oracle Automated Storage Management (Oracle ASM) is preferred storage solution when running Oracle database on Amazon EC2 instance. In Oracle ASM configuration, maximum transactions per minutes on io2 Block express exceeded gp3 by 49% on average.
Stress test
For Oracle ASM configuration, with a simulated load of 150 concurrent sessions to the database for 30 minutes, io2 Block Express completed 54.89% more transactions per second compared to gp3. In addition, io2 Block Express also executed 55.9% higher IOPS than gp3.
Latency test
For Oracle database with ASM configuration, the average latency with 40,000 IOPS running on io2 Block Express volumes was observed to be 0.25 milliseconds. On the other hand, with 40,000 IOPS running on gp3, average disk latency was observed to be 0.478 milliseconds. Amazon EBS io2 Block Express volumes performed 47.85% better than gp3 volumes in terms of disk latency. Amazon EBS io2 Block Express volumes achieved a low time to complete a single I/O operation on a block device with Oracle ASM configuration.
The following image shows the latency during peak load for io2 Block Express:
The following image shows the latency during peak load for gp3:
Cost and performance for gp3 and io2 Block Express
On comparing pricing for Amazon EBS gp3 volumes and io2 Block Express volumes, gp3 volumes incur lower cost with single digit milliseconds latency, whereas io2 Block Express volumes incur higher cost with sub-millisecond latencies.
For the benchmarks performed in this post, the cost on io2 Block Express was 74.57% greater than gp3. However, the average performance improvement achieved by io2 Block Express over gp3 in terms of the number of transactions per min is 52% higher, with 87% lower latency.
| Amazon EBS gp3 | Amazon EBS io2 Block Express | |
| Cost of storage (8 TB for gp3 and 4 TB for io2 Block Express) |
$655.36 | $512 |
| Cost of 80,000 IOPS | $400 | $4,896 |
| Cost of 8000MiB/s Throughput | $320 | – |
| Total cost | $1,375.36 | $5,408 |
Amazon EBS gp3 is a good choice for storage on any database workload that is not sensitive to disk latencies and that do not need sub-millisecond latencies, where cost is a priority over performance. This blog has a good analysis on storage patterns and the analysis on benefits of moving to AWS. This link has further details on benefits of moving from gp2 to gp3 storage.
Amazon EBS io2 Block Express volumes are a great option when performance is a higher priority than cost. They are optimal for databases that have intensive workloads sensitive to disk latencies and which are I/O intensive.
Summary
When running any workload on a database that requires high performance and sub-millisecond latencies, and where performance is a priority over cost, io2 Block Express is the recommended block storage solution to meet your needs.
Amazon EBS io2 Block Express provides advantages of some advanced communication protocols implemented as part of the AWS Nitro System. You can achieve up to 256,000 IOPS and 4000 MBps per volume of throughput with a maximum volume size of 64 TiB, all with low-variance sub-millisecond disk I/O latency.
With Oracle databases running on r5b.24xlarge EC2 instances and configured with Oracle ASM, I compared running databases with Amazon EBS gp3 and Amazon EBS io2 Block Express for storage. I observed that io2 Block Express outperformed gp3 in its transactional and I/O performance.
Based on the tests conducted, the following table summarizes the differences in performance in percentage for each test while running oracle database on io2 Block Express and gp3. The table clearly demonstrates the advantage of using io2 Block Express over gp3 when running high-performance workloads.
| Benefit of io2 Block Express over gp3 |
Load test (transactions per min) |
Stress test (transactions per sec) |
Latency test (milliseconds) |
||||||
| gp3 | io2 Block Express | % Diff | gp3 | io2 Block Express | % Diff | gp3 | io2 Block Express | % Diff | |
| % Difference | 405966 | 603980 | 49% | 11059 | 17131 | 55% | 3.83 | 0.5 | 87% |
I executed the tests in this blog on Oracle databases, however you should also observe similar performance benefits with other databases like Microsoft SQL Server, PostgreSQL, and MySQL.
Thank you for reading this blog post on benefits of using io2 Block Express over gp3 for I/O intensive database workloads. If you have any comments or questions, don’t hesitate to leave them in the comments section.