Developing a Microsoft .NET Core Web API application with Aurora Database using CloudFormation

Real world Microsoft workloads have a lot of Web APIs that are native to Microsoft methods for serving front-end applications (like ASP.NET, ASP.NET Razor/MVC, ReactJS or Angular Application). Even though there are customers who want to try serverless with A WS Lambda, they often have to continue to maintain many existing .NET web APIs. These applications traditionally talk to Microsoft SQL Server database for CRUD operations. This blog is for application teams that are migrating from traditional Microsoft SQL workloads. You can use this blog to learn how to create, connect and integrate with Amazon Aurora Database, while continuing to use existing .NET Core Web APIs.

Many of these existing .NET Core applications can be containerized using docker and AWS services like Amazon EC2/ Amazon Elastic Container Service (ECS), AWS Systems Manager (SSM), Amazon Aurora Database providing a full blown API application system. The application architecture is complemented by build & pipeline tools like AWS CodeCommit, AWS CodeBuild using AWS CloudFormation. At the end of this blog, we will create a simple Microsoft .NET Web API for a ToDo Application. See below, the solution architecture diagram of Microsoft .NET Core Web API Application Orchestration . The sample TODO application will communicate with an Amazon Aurora database for basic CRUD operations. Provided CloudFormation templates will build an ECS with Fargate and a serverless RDS Aurora back-end.

Provided CloudFormation templates have AWS CodeBuild that triggers the committed code (in AWS CodeCommit) and pushes it as an image into Amazon Elastic Container Registry (ECR). The second CloudFormation triggers the ECS Service which has the task definition to read the image from the ECR. The provided .NET project leverages AWS SDK packages to connect to MySQL and interact with Aurora database. The sample exposed has WebAPI endpoint that performs HTTP calls (GET & POST) to add/retrieve TODOs. The end user can use any tool like curl or UI tools like Google Chrome ARC Rest Client or POSTMAN to validate the changes.

Overview of the AWS services used in this solution

Amazon Aurora, a MySQL and PostgreSQL-compatible relational database is used as the backend for the purpose of this project.
Amazon Elastic Container Service (ECS) is used using Docker template containers and allows you to easily run and scale containerized applications on AWS. Sample Dockerfile is provided as part of the solution.
Amazon Elastic Container Registry (ECR), the AWS provided Docker container registry is used and integrated with ECS, simplifying the development to production workflow.
AWS CodeBuild is used to manage the continuous integration of development code into Amazon ECR as a containerized image.
AWS CodeCommit is used as the code repository and integrates with AWS CodeBuild for continuous integration process.

Walkthrough: Microsoft.NET Core Web API Application using Aurora Database

Use the following steps to provision the infrastructure (and services) and deploy the application:

Clone the sample Code and CloudFormation template from this GitHub location. The git source provided above has a “code” and “config” folder. “code” has the necessary .NET Web API solution sample. “config” folder has two AWS CloudFormation templates
1. net-core-task-infrastructure.yaml – This template spins up the necessary hardware/AWS services
2. net-core-stack-services.yaml – This template creates the MySQL ToDos table and also additionally deploys ECS Service with external AWS LoadBalancer URL
3. Sample commands:
  1. $<your_drive>\path\ git clone https://github.com/awslabs/aws-dotnet-webapi-aurora
  2. $ cd aws-dotnet-webapi-aurora
Execute the CloudFormation template using AWS CLI to create the required infrastructure for the application. aws cloudformation create-stack --stack-name net-core-stack --template-body file://config/net-core-task-infrastucture.yaml --capabilities CAPABILITY_NAMED_IAM --parameters ParameterKey=AppStackName,ParameterValue=net-core-stack.
The above command provisions the infrastructure with the required AWS Services including AWS CodeCommit. Once this is complete, push the .NET application code from local machine onto AWS CodeCommit. This will automatically trigger a build using AWS CodeBuild. Once the build is completed, it will push the Docker Image to Amazon ECR.
- Now, get the git clone URL from AWS CodeCommit for the repository and run the following commands to push the application code. Based on the initial CloudFormation stack deployed the AWS CodeCommit repository is created with below format. You may get this from the AWS Console.
  - https://git-codecommit.us-east-1.amazonaws.com/v1/repos/<Stack-Name> todo-repo
- A sample command is as below.$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/net-core-stack-todo-repoClone the provided sample WebAPI .NET code from the provided github repository. Add the .NET Code into your repository like below.
  - git add .
  - git commit -m “initial commit”
  - git push
Execute the following command to deploy the container into the ECS cluster. This CloudFormation stack accepts an EC2 KeyPair that will be used by the EC2 built by the stack. The EC2 instance runs a node script to create the ToDos table in the Aurora Database cluster. Ex: if you already have a KeyPair (my-east1-keypair) the sample command will be as below
$ aws cloudformation create-stack --stack-name net-core-service-stack --template-body file://config/net-core-task-services.yaml --capabilities CAPABILITY_NAMED_IAM --parameters ParameterKey=AppStackName,ParameterValue=net-core-service-stack ParameterKey=KeyName,ParameterValue=my-east1-keypair ParameterKey=StackName,ParameterValue=net-core-stack
To create a new EC2 Key pair refer this link
The below step is completely optional. RDS table is created by the services stack above.
This lets you validate the RDS Database created as part of the CloudFormation. Please review troubleshooting steps provided below for any issues you encounter.
From AWS Console, select RDS > Query editor to select or create a table required for managing the Todos
use todo;
drop table IF EXISTS ToDos;
create table ToDos(
   id MEDIUMINT not null auto_increment,
   CreatedTime TIMESTAMP DEFAULT now(),
   Status VARCHAR(50),
   Task VARCHAR(50),
   primary key(id)
);
select * from ToDos;
The previous step creates the Aurora DB table & ECS Service where the .NET Core Web API is deployed into the ECS Container. The output of the stack returns the following
- HealthCheckUrl– http://<your_ALB>.us-east-1.elb.amazonaws.com/api/values
- WebApiUrl – http:// <your_ALB>.us-east-1.elb.amazonaws.com/api/todo

Stack Verification

Once the above CloudFormation stack is created successfully, take a moment to identify the major infrastructure components like VPC, Public & Private Subnet, Route Tables, Internet Gateway, NAT Gateway, Public Load Balancer, ECS Cluster. Other AWS Services – CodeBuild, CodeCommit, ECR, Amazon Aurora Database (Serverless), Systems Manager, CloudWatch Event Rules & Logs.
All these are built as part of the first CloudFormation stack. Second CloudFormation stack provisions ECS Service, deploys the containers & integrates with the Public Load Balancer. Also, the output of this stack will give the API URLs for health check and API validation.

API Testing

Test the ToDo API using any REST API tools, like Postman, Chrome extension like ARC or RestMan.

GET – Open browser and you can hit the WebApiUrl to see the data.
- http:// <your_ALB>.us-east-1.elb.amazonaws.com/api/todo
POST a sample
- Set Headers as “Content-type” & “application/json”
- Sample request:
```
{
    "Task": "Coding for new project1",
    "Status": "InProgress"
}
```

Use the same url and fire a GET call to see the previously posted todo item as response.

Code Cleanup

Execute the following command to delete the stacks that were created as part of setting up the application.

Using AWS Console, delete the image(s) in the ECR & the contents of the s3 bucket
$ aws cloudformation delete-stack --stack-name net-core-stack-service
$ aws cloudformation delete-stack --stack-name net-core-stack

Note: If you encounter a failure, make sure the contents of the Container registry and S3 buckets are deleted as mentioned above and then execute the above the command.

Optionally you can browse to the AWS Console and delete the stack. Make sure to delete both the stacks to avoid any cost.

Troubleshooting

Make sure to review the AWS service limits. Ex: 5 VPCs per region
After the service stack completes if the WebAPIURL displays blank
- In AWS Console, navigate to EC2 section and make sure the <stack-name>-tablecreatorinstance is running 2/2 complete
  – This EC2 creates the “ToDos” table in the “todo” database in Aurora
  – If issue persist, you can query using RDS Query Editor
- AWS Console Navigate to RDS in AWS Console
  – Validate if the “ToDos” table is created in the Aurora database
  – select your database typically <stack-name>-todo
  – click “Modify”. Scroll down and select “Data API”, Apply immediately in next screen – This lets you to query the DB using AWS RDS Console query editor
  – select “Query Editor” – Provide the connection string. Typically this is available in the first/infrastructure stack
  – in the Query window – execute below SQLs to validate
- use todo;
  select * from ToDos;
- If this doesn’t return records you can manually create the table with below query
- use todo;
  drop table IF EXISTS ToDos;
  create table ToDos(
     id MEDIUMINT not null auto_increment,
     CreatedTime TIMESTAMP DEFAULT now(),
     Status VARCHAR(50),
     Task VARCHAR(50),
     primary key(id)
  );
  select * from ToDos;
- Try hitting the WebApiURL from the browser to see the data.
- Note: Inactivity in Aurora Serverless – RDS Table could put the RDS in suspended state to reduce the cost. You might receive a communication error after no activity while trying to invoke the database DDL/DML statements. You can notice this by connecting to the SQL in Query Editor with below output.Retrying the select queries will warm up the RDS database for subsequent connection to be served.
  - Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

Conclusion

As you see, we were able to onboard an ASP.NET Core Web API application deployed and integrate it with various AWS Services. The post walked through deploying Microsoft .NET Core application code as containers with infrastructure as code. To try out architecture as code with applications on AWS, check out AWS CDK ECS patterns.

We encourage you to test this example and see for yourself how this overall application design works within AWS. Then, it will be just the matter of replacing your individual code base (buildspec/WebAPI, MVC, or other Microsoft .NET core application) and package them as Docker container and let the AWS ECS Container handle the process efficiently.

If you attempt this exercise and have any questions/feedback about this blog please provide your comments below!

References:

About the authors

Sivasubramanian Ramani (Siva Ramani) is a Sr Cloud Application Architect at AWS. His expertise is in application optimization, serverless solutions and using Microsoft application workloads with AWS.

Naveen Balaraman is Cloud Application Architect at Amazon Web Services. He is passionate about application optimization, architecting microservices and helping customers leverage the power of AWS cloud easily.

AWS Developer Tools Blog