Deploy a containerized application with Amazon ECS and connect to Amazon DocumentDB (with MongoDB compatibility) securely

Deploying containerized applications with cloud database services can involve many manual steps, such as configuring networks and securing database connections from the applications. To simplify development and deployment, you can use infrastructure as code (IaC) to build, configure, deploy, and scale containerized applications automatically.

In this post, I show you how to use IaC concepts to deploy a containerized application and connect it to an Amazon DocumentDB (with MongoDB compatibility) cluster with AWS Cloud Development Kit (AWS CDK) and Amazon Elastic Container Service (Amazon ECS).

AWS CDK is an open-source software development framework that enables you to define AWS resources and provision application resources in a safe, repeatable manner with familiar programming languages through AWS CloudFormation.

Amazon DocumentDB is a scalable, highly durable, and fully managed database service for operating mission-critical MongoDB workloads.

Solution overview

The following diagram illustrates our solution architecture. In this architecture, Amazon Virtual Private Cloud (Amazon VPC), a security group, Amazon DocumentDB, and AWS Secrets Manager are configured and deployed with AWS CDK, and container services such as Amazon ECS and AWS Fargate with the Docker Compose CLI. Containerized applications can connect to the database using the output information deployed with AWS CDK.

The containerized applications run on Fargate, a serverless compute engine. Fargate allows you to focus on building applications without managing servers. Amazon ECS is a fully managed container orchestration service, and makes it easier for you to deploy, manage, and scale the containerized applications on Fargate. To protect database credentials securely, the password for authenticating to databases is stored in Secrets Manager. Amazon DocumentDB is deployed in private subnet and can only be accessible through the application with the credentials stored in Secrets Manager for enhancing security.

Prerequisites

Complete the following prerequisite steps:

Install the Docker Compose CLI and create an AWS Context. For more information, see Deploying Docker containers on ECS.
Clone the repository.
Run the following command:
./deployer.sh <AWS_PROFILE>

To learn more about what is being deployed in this solution, refer to How to build smart cities with FIWARE Orion Context Broker and Cygnus on AWS.

Configure Amazon DocumentDB with AWS CDK

This example uses AWS CDK to deploy a VPC and security group, Amazon DocumentDB, and Secrets Manager, as shown in the following diagram. Please see API reference if you would like to use other programming languages.

Define the VPC and security group

You can use the @aws-cdk/aws-ec2 module to define the VPC and security group as shown in the following code. You must define the public subnet where containerized applications are deployed and the private subnet where Amazon DocumentDB is deployed.

The following code creates a VPC with two subnets.

const vpc = new ec2.Vpc(this, "VPC for Orion and Cygnus", {
      cidr: "10.0.0.0/16",
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: "application-subnet",
          subnetType: ec2.SubnetType.PUBLIC,
        },
        {
          cidrMask: 24,
          name: "db-subnet",
          subnetType: ec2.SubnetType.PRIVATE_WITH_NAT,
        },
      ],
    })

Amazon DocumentDB can only be accessible through the same VPC where the container application is deployed. In addition, to enhance security, you can define the Amazon DocumentDB security group to allow only incoming traffic for the containerized applications’ security group. In this solution we defined the database security group to only accept ingress traffic from the application security group (OrionSG) on port 27017 which is the default port of Amazon DocumentDB as follows:

// Documentdb rules: Allow Orion application on container
ddbSG.addIngressRule(orionSG, ec2.Port.tcp(27017));

Define Secrets Manager

You can use the @aws-cdk/aws-secretsmanager module to define the Secrets Manager construct as shown in the following code. You use generateSecretString to define the password condition, such as length, excluding characters and punctuation, so it can be customized according to the application and security requirements.

const ddbPassSecret = new Secret(this,'DocumentDB Password',{
    secretName:'ddbPassword',
    generateSecretString:{
        excludePunctuation:true,
        excludeCharacters:"/¥'%:;{}"
    }
})

Define Amazon DocumentDB

You can use the @aws-cdk/aws-docdb module to define the Amazon DocumentDB construct as shown in the following code. Specify the ARN of Secrets Manager as ddbPassSecret.secretArn, and define the VPC and subnet where you want to deploy the cluster, instance type, and number of instances in the properties. As previously mentioned, you define the private subnet on this construct so that Amazon DocumentDB isn’t accessible via public endpoint.

The following code creates a new DocumentDB cluster attached the VPC and password stored in Secrets Manager which you defined above. Regarding RemovalPolicy, this sample defined DESTROY which means when the resource is removed from the app, it will be physically destroyed. If you deployed DocumentDB in production environment, you can define the options of RETAIN or SNAPSHOT depending on your retention needs. Refer to this document to learn more.

const ddbCluster = new ddb.DatabaseCluster(this,"DDB",{
    masterUser:{
    username:'awsdemo',
    password: SecretValue.secretsManager(ddbPassSecret.secretArn)
    },
    vpc: props.ddbVpc,
    vpcSubnets:props.ddbVpc.selectSubnets({subnetGroupName:'db-subnet'}),
    instanceType: ec2.InstanceType.of(ec2.InstanceClass.R6G, ec2.InstanceSize.XLARGE2),
    instances:2,
    engineVersion: '4.0',
    parameterGroup:parameterGroup,
})

ddbCluster.addSecurityGroups(props.ddbSg)
ddbCluster.applyRemovalPolicy(cdk.RemovalPolicy.DESTROY)

Connect to Amazon DocumentDB from an Amazon ECS application

In this section, I show you how to connect a containerized application to your Amazon DocumentDB cluster.

Define a containerized application on Amazon ECS with a compose file

The containerized application is defined in the following compose file. The following YAML is an excerpt from the definition. In this example, the docker-compose-generator code automatically fills in the variables like <Your VPC-ID> and <Your-docdb-endpoint> from the output created by AWS CDK.

x-aws-vpc: "<Your VPC-ID>"
secrets:
  docdb_password:
    name: "<arn_documentdb_secret>"
    external: true
services:
  # Orion is the context broker
  orion:
    secrets:
      - docdb_password
    image: fiware/orion:latest
    deploy:
      x-aws-autoscaling:
        min: 2
        max: 4 #required
        cpu: 50
      resources:
        limits:
          cpus: "2"
          memory: 2048M
    networks:
      - orion-sg
    container_name: fiware-orion
    ports:
      - "1026:1026"
    entrypoint: ["/bin/sh", "-c"]
    environment: DOCDB_ENDPOINT=<Your-docdb-endpoint>
      DOCDB_USER=awsdemo
      ORION_LOGS_LEVEL=ERROR
    command:
      - |
        sleep 5
        export DOCDB_PASSWORD=`cat /run/secrets/docdb_password`
        /usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -disableFileLog -dbhost $${DOCDB_ENDPOINT}:27017 -rplSet rs0 -dbuser $${DOCDB_USER} -dbpwd $${DOCDB_PASSWORD} -dbDisableRetryWrites -logLevel $${ORION_LOGS_LEVEL}
    healthcheck:
networks:
  orion-sg:
    external: true
    name: <Orion Security Group ID>

Set the secret ARN of Secrets Manager for the DB connection under the name property in secrets. The secrets are mounted to containers. Because the secrets are mounted under /run/secrets/, you can use variables to refer to the secrets value, and use $${variables} to specify the variables in the command as follows:

/usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -disableFileLog -dbhost $${DOCDB_ENDPOINT}:27017 -rplSet rs0 -dbuser $${DOCDB_USER} -dbpwd $${DOCDB_PASSWORD} -dbDisableRetryWrites -logLevel $${ORION_LOGS_LEVEL}

The preceding example passes along the cluster endpoint, port, user name, and password as variables to the FIWARE Orion application. How you specify the connection parameters depends on the programming language you use. To learn more, refer to Connecting Programmatically to Amazon DocumentDB.

Create a Docker context and deploy the application

You can create a Docker context and deploy it locally with the following Docker commands:

docker context create ecs <context-name>? Create a Docker context using: [Use arrows to move, type to filter]
  > An existing AWS profile  <- You can use the profile you created here.

docker context use <context-name>

docker compose -p <your project name> -f <your composefile.yaml> up

The Docker Compose CLI configures Amazon CloudWatch Logs for the container application, and you can check the application logs locally using the logs command as follows:

docker compose --project-name <your project name> logs

To check the status of your application deployed on Amazon ECS, use the ps command as follows:

docker compose --project-name <your project name> ps

#Outputs example
NAME                                          SERVICE             STATUS              PORTS
task/orion/a6033a645faa4cd89c474ba950188ff5   orion               Running             orion-alb-XXXXXXXXXX.us-east-1.elb.amazonaws.com:1026->1026/http
task/orion/d869b3ca63b241c0801c46488d3791ba   orion               Running             orion-alb-XXXXXXXXXX.us-east-1.elb.amazonaws.com:1026->1026/http

Clean up

If you need to clean up the resources, please follow the steps below.

docker compose -p <your project name> down
npm run cdk destroy --all --profile <AWS_PROFILE>

Conclusion

In this post, I showed you how to create an Amazon DocumentDB cluster with AWS CDK and connect to it from a containerized application deployed with Docker Compose. With Amazon DocumentDB, Amazon ECS, and Fargate, you can focus on building containerized applications and business logic without worrying about managing servers, scalability, and availability. The following Docker Compose examples can help you define and deploy your containerized application managed by Amazon ECS. In addition, you can define the deletion protection and rotating credential of Amazon DocumentDB with AWS CDK; see the Amazon DocumentDB Construct Library to learn more.

Try this solution in the GitHub repo, and reach out with your comments or feedback.

About the Author

Hidenori Koizumi is a Prototyping Solutions Architect in Japan’s Public Sector. He is good at developing solutions in the research field based on his scientific background (biology, chemistry, and more). He has recently been developing web applications with React, Recoil, and AWS CDK, and interested in React Location and Rust. He likes traveling and photography.

AWS Database Blog