[SEO Subhead]
This Guidance demonstrates how the unique features of the Amazon S3 Express One Zone storage class can decrease network latency and make the writing of high throughput transactions both instantaneously scalable and fast. S3 Express One Zone is ideal for any application where it's important to minimize the latency required to access an object. This can be human-interactive workflows, like video editing, where creative professionals need responsive access to content from their user interfaces. S3 Express One Zone offers additional advantages to analytics and machine learning workloads that share similar responsiveness requirements from their data, particularly those workloads that require numerous smaller accesses or large numbers of random accesses. It’s compatible with other AWS services that support data analysis as well as artificial intelligence and machine learning (AI/ML) workloads, such as Amazon EMR, Amazon SageMaker, and Amazon Athena.
Please note: [Disclaimer]
Architecture Diagram
[Architecture diagram description]
Step 1
Data engineer creates an Amazon Simple Storage Service (Amazon S3) directory bucket for an application workload.
Step 2
Data engineer creates or configures the appropriate policies for application access to the Amazon S3 Express One Zone storage class.
Step 3
When the application interaction starts, session credentials are generated by an S3 Express One Zone Software Development Kit (SKD), authorized by AWS Identity and Access Management (IAM). The credentials can be used by an application for a limited time period of 5 minutes.
Step 4
High-performance application data persistence write requests, done through Amazon S3 API PUT requests, are directed to the S3 Express One Zone storage class through a gateway endpoint. Object data is available for high-performance query, extract, transform, and load (ETL), compaction, and more.
Step 5
The application object data that needs to be persisted or no longer requires high-performance is copied to an appropriate Amazon S3 storage class using an Amazon S3 Batch Operations copy job.
Step 6
You can delete data no longer needed from the S3 Express One Zone storage class through an authorized application or other options.
Step 1
Data engineer creates an Amazon Simple Storage Service (Amazon S3) directory bucket for an application workload.
Step 2
Data engineer creates or configures the appropriate policies for application access to the Amazon S3 Express One Zone storage class.
Step 3
When the application interaction starts, session credentials are generated by an S3 Express One Zone Software Development Kit (SKD), authorized by AWS Identity and Access Management (IAM). The credentials can be used by an application for a limited time period of 5 minutes.
Step 4
High-performance application data persistence write requests, done through Amazon S3 API PUT requests, are directed to the S3 Express One Zone storage class through a gateway endpoint. Object data is available for high-performance query, extract, transform, and load (ETL), compaction, and more.
Step 5
The application object data that needs to be persisted or no longer requires high-performance is copied to an appropriate Amazon S3 storage class using an Amazon S3 Batch Operations copy job.
Step 6
You can delete data no longer needed from the S3 Express One Zone storage class through an authorized application or other options.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
S3 Express One Zone is a high-performance, single-Availability Zone S3 storage class that is purpose-built to deliver consistent single-digit millisecond data access for your most latency-sensitive applications. S3 Express One Zone can improve data access speeds by 10x and reduce request costs by 50% compared to S3 Standard and scales to process millions of requests per minute for your most frequently accessed datasets. Applications can benefit immediately from requests completing up to an order of magnitude faster. S3 Express One Zone provides similar performance elasticity to other S3 storage classes. As with S3, you don’t need to plan or provision capacity or throughput requirements in advance. You can scale your storage up or down, based on need, and access your data through the Amazon S3 APIs.
-
SecurityRead the Security whitepaper
S3 Express One Zone supports both IAM authorization and session-based authorization. With S3 Express One Zone, you authenticate and authorize requests through a new session-based mechanism, S3 CreateSession, which is optimized to provide the lowest latency. You can use CreateSession to request temporary credentials that provide low latency access to your bucket. These temporary credentials are scoped to a specific S3 directory bucket. Once authenticated with CreateSession, data in the S3 Express One Zone storage class is strongly consistent and always stored in S3 directory buckets, with S3 Block Public Access always enabled on directory buckets. All data stored in S3 Express One Zone is automatically encrypted using Server-Side Encryption (SSE-S3).
-
ReliabilityRead the Reliability whitepaper
S3 Express One Zone is one of several S3 storage classes that you can choose from based on the data access, data redundancy, and cost requirements of your workloads. S3 Express One Zone is designed for 99.95% availability within a single Availability Zone backed by the Amazon S3 Service Level Agreement. S3 Express One Zone objects are stored in a single AWS Availability Zone that you choose. In the unlikely case of the loss or damage to all or part of an Availability Zone, data in an S3 Express One Zone storage class may be lost. For example, events like fire and water damage could result in data loss. Apart from these types of events, our S3 Express One Zone storage classes use similar engineering designs as our Regional storage classes to protect objects from independent disk, host, and rack-level failures, and each are designed to deliver 99.999999999% data durability.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
S3 Express One Zone uses a unique architecture to optimize for performance and deliver consistently low request latency. S3 Express One Zone stores data on high-performance hardware, and its object protocol has been enhanced to streamline authentication and metadata overhead. While you have always been able to choose a specific AWS Region to store your S3 data, with S3 Express One Zone you can select the specific AWS Availability Zone within an AWS Region to store your data. You can choose to co-locate your storage with your compute resources in the same Availability Zone to further optimize performance, which helps lower compute costs and run workloads faster. Additionally, to further increase access speed and support hundreds of thousands of requests per second, data is stored in a new bucket type: an Amazon S3 directory bucket. Each directory bucket can support hundreds of thousands of transactions per second (TPS), regardless of the key names or access patterns.
-
Cost OptimizationRead the Cost Optimization whitepaper
S3 offers a range of storage classes that you can choose from based on the data access, data redundancy, and the cost requirements of your workloads. S3 Express One Zone does not require pre-provisioning, so you only pay for what you use. It will optimize storage for high-performance workloads, reduce timeout (503 responses), expedite queries, and reduce your compute costs for performance demanding applications. S3 Express One Zone is built for high transaction rates and typically has lower API request costs than other S3 storage classes. The volume of storage billed in a month is accrued based on total storage used per hour, measured in gigabyte per month (GB-Month). Request costs are based on the size of the request. You are charged a per-request fee for access based on the request type—such as PUTs and GETs. An additional per-GB fee will be charged for the portion of the request size exceeding 512 KB.
-
SustainabilityRead the Sustainability whitepaper
S3 Express One Zone helps you to scale your storage up or down based on your requirements. You no longer need to manage multiple storage systems for low latency workloads because your applications access data through the same S3 APIs you use today. S3 Express One Zone can grow to petabyte scale, supporting any size workload while keeping a minimal resource footprint. S3 directory buckets can scale up to hundreds of thousands of transactions per second to meet your most demanding workloads. It also offers faster, more scalable storage performance that can reduce CPU and GPU idle time from CPU or GPU idle workloads, optimizing your compute costs.
Implementation Resources
A detailed guide is provided to experiment and use within your AWS account. Each stage of building the Guidance, including deployment, usage, and cleanup, is examined to prepare it for deployment.
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
[Title]
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.