DynamoDB FSI Service Spotlight

Introduction to DynamoDB

In this edition of the Financial Services Industry (FSI) Services Spotlight monthly blog series, we highlight five key considerations for customers running workloads on Amazon DynamoDB: achieving compliance, data protection, isolation of compute environments, audits with APIs, and access control/security. Across each area, we will examine specific guidance, suggested reference architectures, and technical code to help streamline service approval of Amazon DynamoDB.

DynamoDB is a fast and flexible NoSQL database service for any scale. DynamoDB enables customers to offload the administrative burdens of operating and scaling distributed databases to AWS so that they don’t have to worry about hardware provisioning, setup and configuration, throughput capacity planning, replication, software patching, or cluster scaling. DynamoDB takes away one of the main stumbling blocks of scaling databases: the management of database software and the provisioning of the hardware needed to run it. You can deploy a NoSQL database in a matter of minutes. DynamoDB automatically scales throughput capacity to meet workload demands, and partitions and repartitions your data as your table size grows. Also, DynamoDB synchronously replicates data across three availability zones in an AWS Region, giving you high availability and data durability.

DynamoDB supports both key-value and document data models. This enables DynamoDB to have a flexible schema, so each row can have any number of columns at any point in time. This allows you to easily adapt the tables as your business requirements change, without having to redefine the table schema as you would in relational databases. DynamoDB stores data in partitions. A partition is an allocation of storage for a table, backed by solid state drives (SSDs) and automatically replicated across multiple Availability Zones within an AWS Region. Partition management is handled entirely by DynamoDB. This enables DynamoDB to scale to more than 10 trillion requests per day with peaks greater than 20 million requests per second, over petabytes of storage¹. DynamoDB Accelerator (DAX) is an in-memory cache that delivers fast read performance for your tables at scale. Using DAX, you can improve the read performance of very frequently accessed items in your DynamoDB tables by up to 10 times—taking the time required for reads from milliseconds to microseconds, even at millions of requests per second. DynamoDB global tables replicate your data automatically across your choice of AWS Regions, as result your globally distributed applications can access data locally in the selected Regions to get single-digit millisecond read and write performance, and be protected from regional failure with zero RTO and near zero RPO.

DynamoDB can provide the same consistent performance at any scale, so you won’t need to refactor your database as your data grows. DynamoDB is built for mission-critical workloads, including support for atomicity, consistency, isolation, and durability (ACID) transactions for a broad set of applications that require complex business logic. DynamoDB helps secure your data with encryption and continuously backs up your data for protection, while providing availability through a service level agreement.

Today, there are numerous FSI customers leveraging Amazon DynamoDB for their use cases. Experian migrated off of Microsoft SQL server to Amazon DynamoDB, resulting in their ability to build flexible and reusable microservices to segment data using Amazon DynamoDB as a serverless data store for consistent and fast performance. Since Experian launched its consumer services platform on Amazon DynamoDB, the company has handled 50–75 percent volume growth in its data layers each year—and with a more flexible cost model, buying on-demand instead of making large capital investments in hardware, software, storage, and networking. State Auto, a property and casualty insurer operating in 33 US states, provides insurance products covering personal, commercial, farm and ranch, and workers’ compensation risks. In 2019, State Auto decided to create a cloud-native version of its traditional technology stack using Amazon Web Services (AWS). State Auto was looking to get more developers hands away from keyboards when developing a solution, meaning they were looking for managed and easy-to-use cloud services. State Auto adopted Amazon DynamoDB for their application, and the managed-services features of DynamoDB have allowed their teams to achieve a product time decrease from months to as little as 24 hours, which means that State Auto can release new features weekly or even daily.

Achieving compliance

Security and compliance are a shared responsibility between AWS and the customer. AWS will operate, manage, and protect the infrastructure that runs the AWS services. The customer’s responsibility is determined by the service selected; the more managed services are used, the less customer configuration is required. As Amazon DynamoDB is a managed service, customers are responsible for fewer controls to deploy secure transactional workloads. On the customer’s side of the shared responsibility model, customers should first determine their requirements for network connectivity, encryption, and access to other AWS resources. We will dive deeper into those topics in the upcoming sections.

DynamoDB falls under the scope of the following compliance programs regarding AWS’s side of the shared responsibility model. The compliance programs covered by DynamoDB include:

SOC 1, 2, 3
PCI
ISMAP
FedRAMP Moderate (E/W) and FedRAMP high (GovCloud)
DoD CC SRG IL2 through IL6
HIPAA BAA
IRAP
MTCS
C5
K-ISMS
ENS High
OSPAR
HITRUST CSF
FINMA
GSMA
PiTuKri

Logging and Auditing

You could use AWS CloudTrail to log both control plane activity on your DynamoDB tables and data plane activity logging for fine-grained monitoring of all DynamoDB item activity within a table. The control plane activity can be creating or deleting a table, while the data plane activity can be putting or getting items in the table. You can find more about the DynamoDB operation logging in the DynamoDB control plane and data plane events in CloudTrail.

If you’re a database administrator or security professional, you can use this information as part of an audit. Each event carries information, such as who performed an action and when, which resources were impacted, and many other details. Events are combined in JSON format and saved in CloudTrail log files. You can find sample event JSON files in this link.

Data protection

Security is the top priority at AWS, DynamoDB encrypts user data stored at rest and also data in transit.

Encryption At Rest

Encryption at rest is crucial for regulatory compliance to ensure that only authorized principals can access the sensitive data on underlying storage with a valid key. Amazon DynamoDB encryption at rest encrypts your data using 256-bit Advanced Encryption Standard (AES-256). When you access an encrypted table, DynamoDB decrypts the table data transparently with minimal impact on performance. You don’t need to modify your applications to use encryption.

On-demand backups are encrypted with the table-level encryption key that is specified at the time the backup is created. Updating the table-level encryption key does not change the encryption key that is associated with existing on-demand backups.

When creating a new table, you can choose one of the following AWS KMS keys to encrypt your table:

encryption at rest

Financial services customers often require to use the customer-managed key. When you specify a customer-managed key as the table-level encryption key, the DynamoDB table, local and global secondary indexes, and streams are encrypted with the same customer-managed key. You can refer to the blog for details how to use the customer-managed key to encrypt a new table or existing tables.

AWS Config provides the managed rules dynamodb-table-encrypted-kms command to check if Amazon DynamoDB table is encrypted with AWS KMS.

Encryption In Transit

By default, communications to and from DynamoDB use the HTTPS protocol, which protects network traffic by using Transport Layer Security (TLS) encryption. AWS recommends accessing the AWS published DynamoDB API over TLS 1.2 or above.

You can also use a virtual private cloud (VPC) endpoint for DynamoDB to enable Amazon EC2 instances in your VPC to use their private IP addresses to access DynamoDB with no exposure to the public internet. You can see the detailed explanation of the VPC endpoint for DynamoDB in the section Network Traffic Isolation below.

DAX Data Protection

Amazon DynamoDB Accelerator (DAX) delivers fast response times for use cases that require response times in microseconds. DAX runs within an Amazon Virtual Private Cloud (Amazon VPC). Even though Amazon VPC provides a trust boundary, you can enable an additional layer of data protection both at rest and in transit when creating a new DAX cluster as below:

encryption

Same as DynamoDB, DAX uses AES-256 to encrypt stored data. DAX uses the AWS managed key. After your data is encrypted, DAX handles the decryption of your data transparently with minimal impact on performance.

The traffic between your application and your DAX cluster remains in your Amazon VPC. DAX encryption in transit adds to this baseline level of confidentiality, ensuring that all requests and responses between the application and the cluster are encrypted by transport-level security (TLS), and connections to the cluster can be authenticated by verification of a cluster x509 certificate. The DAX client will use the cluster’s x509 certificate to authenticate the identity of the cluster when it establishes connections.

You can find more Operational Best Practices for Amazon DynamoDB.

Isolation of compute environments

DynamoDB automatically spreads the data and traffic for a table over a sufficient number of servers to handle the request capacity the customer specified and the amount of data stored, while maintaining consistent, fast performance. All data items are stored on Solid State Drives (SSDs) and are automatically replicated across 3 availability zones in a region to provide built-in high availability and data durability.

To control who can use the DynamoDB resources and API, you can set up permissions in AWS IAM. In addition to controlling access at the resource level, IAM allows you to control access at the database level — you can create IAM policies (identity-based policies) that allow or deny access to tables, indexes, streams, items (rows) and attributes (columns) based on the needs of your application.

Network Traffic Isolation

Customers have two connectivity options between their private network and AWS: AWS Site-to-Site VPN or an AWS Direct Connect connection. Access to DynamoDB via the network is through AWS published APIs. The connections could use HTTPS when you use the AWS SDK. You can use following conditional policy to allow HTTPS only request.

{

“Effect”: “Deny”,
….
“Condition”: {

“Bool”: {
“aws:SecureTransport”: “false”

}

For security reasons, many AWS customers run their applications within an Amazon Virtual Private Cloud environment (Amazon VPC). With Amazon VPC, you can launch Amazon EC2 instances into a virtual private cloud, which is logically isolated from other networks—including the public internet. With an Amazon VPC, you have control over its IP address range, subnets, routing tables, network gateways, and security settings.

To access the public internet, your VPC must have an internet gateway—a virtual router that connects your VPC to the internet. This allows applications running on Amazon EC2 in your VPC to access internet resources, such as Amazon DynamoDB.

By default, communications to and from DynamoDB use the HTTPS protocol, which protects network traffic by using SSL/TLS encryption. The following diagram shows how an EC2 instance in a VPC accesses DynamoDB:

Many customers prefer to keep their network traffic isolated to their private network. You can address these concerns by using a virtual private network (VPN) to route all DynamoDB network traffic through your own corporate network infrastructure. However, this approach can introduce bandwidth and availability challenges.

VPC endpoints for DynamoDB can alleviate these challenges. A VPC endpoint for DynamoDB enables Amazon EC2 instances and Lambda functions in your VPC to use their private IP addresses to access DynamoDB with no exposure to the public internet. Your EC2 instances do not require public IP addresses, and you don’t need an internet gateway, a NAT device, or a virtual private gateway in your VPC. You use endpoint policies to control access to DynamoDB. Traffic between your VPC and the AWS service does not leave the Amazon network. The following diagram shows how an EC2 instance or Lambda function in a VPC can use a VPC endpoint to access DynamoDB.

vpc endpoints For more information, see Tutorial: Using a VPC Endpoint for DynamoDB.

Automating audits with APIs

AWS Config monitors the configuration of resources and can send alerts in case resources fall into a non-compliant state. The service provides the ability to use predefined AWS managed rules or define custom AWS Lambda-based rules to monitor compliance, autoscaling, backups, or encryption standards. Example of AWS managed rules:

Besides managed rules in Config, customers can build custom Config rules using API calls related to Amazon DynamoDB recorded by AWS CloudTrail. For a complete list of DynamoDB APIs review the Amazon DynamoDB API References. Following is an example of what a CloudTrail log looks like for the CreateTable API:

{
    "eventVersion": "1.08",
    "userIdentity": {
        ...
    },
    "eventTime": "2022-03-21T16:28:23Z",
    "eventSource": "dynamodb.amazonaws.com",
    "eventName": "CreateTable",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "192.0.2.0",
    "userAgent": "aws-cli/1.15.42 Python/3.6.1 Darwin/17.7.0 botocore/1.10.42",
    "requestParameters": {
        "attributeDefinitions": [
            ...
        ],
        "tableName": "TestTable",
        "keySchema": [
            ...
        ],
        "billingMode": "PROVISIONED",
        "provisionedThroughput": {
            "readCapacityUnits": 5,
            "writeCapacityUnits": 5
        }
    },
    "responseElements": {
        "tableDescription": {
            "tableId": "8aa81c02-39e1-4af3-80e4-2677361d6651",
            "attributeDefinitions": [
                ...
            ],
            "tableName": "TestTable",
            "itemCount": 0,
            "creationDateTime": "Mar 21, 2022, 4:28:23 PM",
            "provisionedThroughput": {
                "readCapacityUnits": 5,
                "writeCapacityUnits": 5,
                "numberOfDecreasesToday": 0
            },
            "tableSizeBytes": 0,
            "tableStatus": "CREATING",
            "keySchema": [
                ...
            ],
            "tableArn": "arn:aws:dynamodb:us-east-1:ACCOUNT_ID:table/TestTable"
        }
    },
    "requestID": "2GUB2KFFFKDJKIQ2F72VKMRR07VV4KQNSO5AEMVJF66Q9ASUAAJG",
    "eventID": "074eda2a-c373-4e9b-83c5-3c9a9c7cf739",
    "readOnly": false,
    "resources": [
        {
            "accountId": "ACCOUNT_ID",
            "type": "AWS::DynamoDB::Table",
            "ARN": "arn:aws:dynamodb:us-east-1:814764508123:table/TestTable"
        }
    ],
    "eventType": "AwsApiCall",
    "apiVersion": "2012-08-10",
    "managementEvent": true,
    "recipientAccountId": "ACCOUNT_ID",
    "eventCategory": "Management",
    "sessionCredentialFromConsole": "false"
}

AWS Audit Manager helps FSI customers continuously audit their AWS usage and simplify how they assess risk and compliance with regulations and industry standards. AWS Audit Manager collects and organizes the evidence by selected frameworks such as PCI-DSS, SOC 2, and GDPR from different sources (including AWS CloudTrail) to compare the environment’s configurations against the compliance controls. Audit Manager saves time with an automated collection of evidence and provides audit-ready reports for customers to review, these reports use cryptographic verification to ensure their integrity.

Operational access and security

Identity-based Policy

You need an IAM user or IAM role to authenticate your DynamoDB access. Using identity-based policies (IAM policies), you can grant permissions to a person or group in their account to create or access Amazon DynamoDB resources, such as reading from an Amazon DynamoDB table. DynamoDB primary resources are tables, indexes, and streams.

For each DynamoDB resource, the service defines a set of API operations. To grant permissions for these API operations, DynamoDB defines a set of actions that you can specify in a policy. For a list of available operations, see Amazon DynamoDB Actions.

AWS customers in the financial services industry may require its trading desk could only access its own trades within the Trades table. The following is an example of identity-based policy that grants permissions for the read-only actions on his/her own trades. You can find more DynamoDB custom policy examples here.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowReadOnlyAccessToOnlyTradesMatchingTraderID",
            "Effect": "Allow",
            "Action": [
                "dynamodb:GetItem",
                "dynamodb:BatchGetItem",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:ConditionCheckItem"
            ],
            "Resource": [
                "arn:aws:dynamodb:us-east-1:ACCOUNT_ID:table/Trades"
            ],
            "Condition": {
                "ForAllValues:StringEquals": {
                    "dynamodb:LeadingKeys": [
                        "${www.amazon.com:user_id}"
                    ],
                    "dynamodb:Attributes": [
                        "TraderId",
                        "TradeId",
                        "Stock",
                        "Price",
                        "Quantity",
                        "TradeDateTime"
                    ]
                }, "StringEqualsIfExists": {
                    "dynamodb:Select": "SPECIFIC_ATTRIBUTES" 
                }
            }
        }
    ]
}

To help our customers avoid having to investigate which permissions are needed for some common use cases, AWS provides the following AWS managed policies:

AmazonDynamoDBReadOnlyAccess – Grants read-only access to DynamoDB resources.
AmazonDynamoDBFullAccess – Grants full access to DynamoDB resources.

It’s important to implement IAM policies that follow the principle of least privilege and enforce separation of duties with the appropriate authorization for each role. The typical roles can be:

DynamoDB administrators, who act like DBA that manages the DynamoDB permissions.
Application developers, who need to read or write DynamoDB for business applications.
AWS resources, which need to access DynamoDB on behalf of the application running inside. Typical examples are EC2 instances, ECS tasks, EKS pods, and Lambda functions.

DAX Identity-based Policy

DAX provides a caching layer to your time-sensitive applications. DAX and DynamoDB have separate access control mechanisms.

When you create a DAX cluster, you must associate the cluster with an IAM role, called service role. The service role prescribes which DynamoDB tables or actions the DAX cluster can apply. DAX does not enforce user-level separation. All the users who can access the DAX cluster inherit the permissions of the service role. If you require isolation, you can create different DAX clusters with different service roles, and allow the right person or application to access the right DAX cluster.

Logging in DynamoDB

Regulators may require our FSI customers to keep access audit of sensitive data for a certain period. AWS CloudTrail captures all API calls for DynamoDB as events, including calls from both DynamoDB console and applications, so we strongly recommend you create a CloudTrail trail to enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for DynamoDB in order to address audit needs.

Please note that you need to enable logging of data plane API activity in CloudTrail for DynamoDB if you need to audit Amazon DynamoDB object-level API activity on tables or streams.

Conclusion

In this post, we reviewed Amazon DynamoDB, highlighting essential information that can help FSI customers accelerate the service’s approval within these five categories: achieving compliance, data protection, isolation of computing environments, automating audits with APIs, and operational access and security. While not a one-size-fits-all approach, the guidance can be adapted to meet the organization’s security and compliance requirements.

Be sure to visit the FSI Service Spotlight Blog Series to learn how FSI customers are using other AWS services from a security lens. You may also find the following additional resources useful:

AWS Security Documentation
The security documentation repository shows how to configure AWS services to help meet security and compliance objectives. Cloud security at AWS is the highest priority. AWS customers benefit from a data center and network architecture that are built to meet the requirements of the most security-sensitive organizations.
AWS Compliance Center
The AWS Compliance Center is an interactive tool that provides customers with country-specific requirements and any special considerations for cloud use in the geographies in which they operate. The AWS Compliance Center has quick links to AWS resources to help with navigating cloud adoption in specific countries, and includes details about the compliance programs that are applicable in these jurisdictions. The AWS Compliance Center covers many countries and more countries continue to be added as they update their regulatory requirements related to technology use.
AWS Well-Architected Framework and AWS Well-Architected Tool
The AWS Well-Architected Framework helps customers understand the pros and cons of decisions they make while building systems on AWS. The AWS Well-Architected Tool helps customers review the state of their workloads and compares them to the latest AWS architectural best practices. For more information about the AWS Well-Architected Framework and security, see the Security Pillar – AWS Well-Architected Framework whitepaper. For best practices for financial services, please see the Financial Services Industry Lens – AWS Well-Architected Framework.

¹ During Prime Day 2021, DynamoDB peak API calls reached 89.2 million requests per second. https://thinkwithwp.com/blogs/aws/prime-day-2021-two-chart-topping-days/

AWS for Industries