.NET observability with Amazon CloudWatch Application Signals

This blog post provides a detailed walkthrough on integrating Amazon CloudWatch Application Signals with .NET applications deployed on an Amazon Elastic Kubernetes Service (EKS) cluster. The solution uses CloudWatch Observability Add-On for EKS to enable the .NET applications to emit telemetry signals using OpenTelemetry automatically.

Introduction

Amazon Web Services (AWS) announced Amazon CloudWatch Application Signals during re:Invent 2023. It is a new feature for monitoring and understanding the health of applications. Today, we are excited to announce that Application Signals now supports .NET. Adding Application Signals to your .NET workload enables you to use AWS Distro for OpenTelemetry (ADOT) to auto-instrument your .NET applications without code changes, which allows your application to start emitting Metrics and Trace.

Application Signals will automatically collect these metrics and traces from your applications and display key metrics such as call volume, availability, latency, faults, and errors. It also generates operational dashboards, which empower you to visualize and triage operational health quickly and whether your applications meet their longer-term performance goals without you writing custom code or creating a dashboard.

Solution overview

Consider a containerized .NET workload (Figure 1) on an Amazon Elastic Kubernetes Service (EKS) cluster. It comprises two .NET Web API applications, one emulating a Cart API and the other emulating a Delivery API. When a user requests cart details, the Cart API gets the data from its DynamoDB table, and it uses the .NET built-in HTTP Client library to fetch the delivery data from the Delivery API. Although this example solution represents only two services communicating via HTTP calls, in modern application architectures such as distributed systems or microservices, it is common to encounter systems or platforms with dozens, hundreds, or even thousands of microservices that communicate using HTTP, Events, Messaging, Topics, Queues, Web socket or gRPC. They also use a variety of databases, frameworks, and libraries, which add complexity and challenges to operations.

To operate a containerized .NET workload like these, or more complex ones, you need to instrument each microservice and its components to emit telemetry data such as Metrics, Traces, and Logs, correlate those telemetry data, create dashboards, alarms, and monitoring capabilities. Application Signals helps you create those operational capabilities and reduce their implementation complexity by automatically discovering and presenting the topology map of your application, dependencies, and connectivity. You can also easily create and track the status of service-level objectives (SLOs) related to CloudWatch metrics, including the new standard application metrics that Application Signals collect. This example solution provides a guide on implementing Application Signals for .NET applications on EKS using CDK as infrastructure-as-code.

Figure 1 – Architecture diagram

Prerequisites

For the code example, use the GitHub repository “/aws-samples/dotnet-observability-cloudwatch-application-signals,” which contains a sample code for two microservices. We define and provision the Infrastructure as Code (IaC) using the AWS Cloud Development Kit (CDK) with the TypeScript programming language.

To run this sample, you will need the following prerequisites on your system:

• .NET 8 Software Development Kit (SDK)
• AWS Cloud Development Kit (AWS CDK) v2
• Cloud Development Kit for Kubernetes (CDK8s)
• Docker Engine
• Visual Studio Code (or your preferred IDE)
• AWS Command-Line Interface (AWS CLI)
• AWS credentials for your AWS account configured locally
• Git
• NodeJS (LTS)

Clone the sample repository from the command line with the following command:

git clone https://github.com/aws-samples/dotnet-observability-cloudwatch-application-signals.git

Deploy the EKS cluster using CDK

We built the EKS Cluster using the Amazon EKS Blueprints Quick Start with CDK for this example solution. We also enable the CloudWatch EKS Add-On, which allows us to enable Application Signals to collect telemetry from the EKS Cluster workloads. The following code snippet outlines how we define the EKS Cluster.

export class EksBlueprintStack {
  constructor(scope: Construct, id: string) {
    ...
    const addOns: Array<blueprints.ClusterAddOn> = [
      ...
      new blueprints.addons.AdotCollectorAddOn(),
      new blueprints.addons.CloudWatchAdotAddOn(),
      ...
    ];
    const stack = blueprints.EksBlueprint.builder()
      .version(KubernetesVersion.V1_30)
      .addOns(...addOns)
      .useDefaultSecretEncryption(false) 
      .build(scope, id);
    ...
  }
}

Run the following command to navigate to the IaC folder and deploy the EKS Cluster, Cart DynamoDB table, and the two .NET application container images to Amazon Elastic Container Registry (Amazon ECR).

cd src/iac/aws
npm ci
cdk deploy --require-approval never demo-app-stack eks-blueprint-demo

Configure kubectl

Run the following shell command to add the EKS cluster endpoint to your local kubeconfig file. This will allow you to interact with the Kubernetes cluster using kubectl on your local machine.

eval $(aws cloudformation describe-stacks  --stack-name eks-blueprint-demo --output text --query 'Stacks[0].Outputs[?contains(OutputKey,`eksblueprintdemoConfigCommand`)].OutputValue  | [0]')

Deploy Kubernetes resources using CDK8s

To enable Application Signals for .NET applications, you must add an annotation instrumentation.opentelemetry.io/inject-dotnet: ‘true’ to a manifest YAML in the cluster. Adding this annotation auto-instruments the application to send metrics and traces to Application Signals. In this example solution, we use CDK8s to define the Kubernetes Resources using TypeScript instead of YAML. The following code snip is an example of how to add the Application Signals annotations.

const appDeployment = new kplus.Deployment(this, "web-app", {
  serviceAccount: appServiceAccount,
  podMetadata: {
    annotations: {
      "instrumentation.opentelemetry.io/inject-dotnet": "true",
      "instrumentation.opentelemetry.io/otel-dotnet-auto-runtime": "linux-musl-x64",
    },
  },
  containers: [
    {
      image: props.image,
      args: props.args ?? [],
      portNumber: props.portNumber,
      envVariables: props.envVariables ?? undefined,
      ...
    },
  ],
});

In a terminal, from the root folder where you’ve cloned the repository, perform the following steps:

• Navigate to the CDK8s infrastructure folder
• Get the container image URL from the ECR Registry for the two .NET Application
• Synthesize the code to generate the CloudFormation files
• Deploy Kubernetes resources using kubectl

# navigate to IaC folder
cd src/iac/k8s

#get container image URL
export CART_IMAGE=$(aws cloudformation describe-stacks  --stack-name demo-app-stack --output text --query 'Stacks[0].Outputs[?contains(OutputKey,`CartDockerImageUri`)].OutputValue  | [0]')
export DELIVERY_IMAGE=$(aws cloudformation describe-stacks  --stack-name demo-app-stack --output text --query 'Stacks[0].Outputs[?contains(OutputKey,`DeliveryDockerImageUri`)].OutputValue  | [0]')

#Synthesize cdk8s
npm ci
cdk8s synth

#deploy kubernetes resources
kubectl apply -f ./dist/

Verify deployment results

Run the following kubectl command to confirm the successful deployment of application resources. It should display outputs similar to Figure 2.

kubectl get pods

Figure 2 – Get pods outputs

Create CloudWatch Synthetics canaries to generate traffic.

Next, you will create canaries by running the following commands, which deploy a CloudWatch Synthetic Canary that runs every 2 minutes to generate traffic for the Cart application. From the root folder where you cloned the project, run the following.

cd src/iac/aws
export EKS_URL_BASE=$(kubectl get ingress | awk 'NR==2 {print $4}')
export CART_WORKLOAD_NAME=$(kubectl get deployments | awk 'NR==2 {print $1}')

cdk deploy --require-approval never demo-canary-stack

Visualizing application using CloudWatch Application Signals

Wait 6 to 10 minutes for the Synthetic canary to generate traffic loads (Figure 3). Then, you can navigate to Amazon CloudWatch to review the services Application Signals discovered, the metrics and traces, the dashboard it creates automatically, and more.

Services dashboard

CloudWatch Application Signals automatically discover and show a list of services requiring no additional setup under the Services dashboard out-of-the-box. This unified, application-centric view helps provide a complete perspective of how users interact with your service, which can help you triage issues if performance anomalies occur.

Figure 3 – Service dashboard

Detailed service information and dependencies

The Service detail page shows an overview of your services, operations, dependencies, canaries, and client requests for a single service enabled for Application Signals. To view the service page (Figure 4), do the following:

1. Open the CloudWatch console.
2. In the left navigation pane, choose Application Signals and then select Services.
3. Select the service’s names from the Services table, such as “cart-web-app***” (Figure 2).

The Service Overview (Figure 4) summarizes your service’s components and highlights key performance metrics to help you identify issues that require troubleshooting.

Figure 4 – Service overview

Application Signals uses the Distributed Trace telemetry generated from the applications to correlate with operational metrics and provide a single view to help you quickly find the traces related to a point in the metrics chart. For example, suppose you notice something abnormal or relatively high on the metrics chart. From the Service Overview (Figure 4), choose the tab “Service Operation” you should get a page like (Figure 5). Then you can:

1. Select the operation (e.g. “GET /apps”).
2. Select the height point in the metrics chart.
3. Application Signals will display a panel (Figure 5) with Correlated Trace, Top Contributor, and Application logs. You can then select the corresponding trace to farther investigate.

Figure 5 – Service operations

From the “Correlated Traces” panel, you can select one of the listed Trace IDs. The console will redirect you to the AWS X-Ray Console page (Figure 6) with more details about the selected Trace ID.

Figure 6 – Distributed trace details

Service map

Open the CloudWatch console to explore a Service Map with your application topology. In the left navigation pane, select the Application Signals section and choose a Service Map. The console will present a map of your services. Choose one service, for example, “cart-web-app-***,” to show a panel (Figure 7) with key metrics for that service.

Figure 7 – Service map

Service level objectives (SLOs)

Use Application Signals to establish service level objectives (SLOs) for your critical business operations. By defining SLOs for these services, you can monitor them on the SLO dashboard, providing a quick overview of your crucial operations. SLO conditions contain metrics for latency and availability and CloudWatch metrics, offering comprehensive tracking capabilities.

Follow the Create an SLO step to create SLOs for the Cart application (Figure 8).

Figure 8: Create and visualize service level objectives (SLOs)

Cleanup

You should clean up the resources created by running this demo to avoid unexpected charges. To do so, run the following commands from the root folder where you’ve cloned the GitHub repository.

cd src/iac/k8s
kubectl delete -f dist/
cd ../aws   
cdk destroy --require-approval never --all   

Conclusion

In this blog post, we showed you how to enable Amazon CloudWatch Application Signals for .NET applications running on Amazon EKS. Application Signals provides comprehensive Observability of your .NET services and automatic instrumentation without code changes. It discovers application components, generates operational dashboards with key metrics, visualizes service dependencies, correlates metrics, traces, and logs all out-of-the-box.
With Application Signals, you get an end-to-end view of the health and performance of your .NET applications. Service maps, detailed metrics, correlated traces, and service level objectives give you the insights to quickly triage issues, optimize performance, and ensure your services meet business requirements. Application Signals simplify Observability significantly, allowing teams to focus more on innovation.

Please refer to the CloudWatch Application Signals documentation for more information, or the guide to enable Application Signals on Amazon EC2, Amazon ECS, or the Observability Workshop for hands-on experience.

AWS has significantly more services, and more features within those services, than any other cloud provider, making it faster, easier, and more cost effective to move your existing applications to the cloud and build nearly anything you can imagine. Give your Microsoft applications the infrastructure they need to drive the business outcomes you want. Visit our .NET on AWS and AWS Database blogs for additional guidance and options for your Microsoft workloads. Contact us to start your migration and modernization journey today.