Collect custom metrics with Amazon CloudWatch strategic tagging

If you deploy multiple applications to your Amazon Web Services account, then your organization is likely interested in collecting metrics based on the varied purposes of those applications. For example, consider an environment that runs two Amazon Elastic Compute Cloud (Amazon EC2) instances. The first instance hosts a front-end application, and the second hosts a database application. Disk-usage metrics can be valuable for the database application but not the front-end application.

Manually configuring and collecting custom metrics can add to administrative workloads, especially if you plan to add more applications over time.

In this post, we explain how to use an Amazon CloudWatch agent to automate the collection of custom metrics for two Amazon EC2 instances. The solution is based on a tagging strategy combined with AWS Systems Manager State Manager to enforce specific CloudWatch agent configurations for EC2 instances that are tagged appropriately. After we configure the automation, we show you how to the launch the EC2 instances and confirm that the metrics are successfully being sent to the Amazon CloudWatch Logs.

To save you time, we provide a deployable CloudFormation template with preconfigured resources and an AWS Identity and Access Management (IAM) role and instance profile with the required permissions.

Overview

Figure 1 shows the workflow of using two Amazon EC2 instances.

Figure 1: Automation workflow

1. AWS Systems Manager Parameter Store to store the Amazon CloudWatch agent JSON configurations.
2. Parameter Store passes the JSON configurations to the Systems Manager command document via a parameter.
3. AWS Systems Manager State Manager uses the command document to identify Amazon EC2 instances based on specific tags.
4. Command document installs CloudWatch agent on the new EC2 instances using the JSON configurations stored in Parameter Store.
5. Amazon EC2 instances are launched and tagged.
6. Custom metrics that are gathered from the EC2 instances are delivered to CloudWatch.

Process overview

In the exercises that follow, you perform the following steps:

1. For each Amazon EC2 instance, configure CloudWatch agents in a JSON configuration file and store it in the Parameter Store.
2. Create a command document to install and configure CloudWatch agents that are configured in the JSON files.
3. Create an AWS Systems Manager State Manager association that detects when EC2 instances with ApplicationA and ApplicationB tags are launched.
4. Launch the two EC2 instances with the ApplicationA and ApplicationB tags.
5. Verify that the new instances are associated with State Manager.
6. Verify that the custom metrics are sent to the Amazon CloudWatch Logs. The EC2 instance tagged with ApplicationA tracks the percentage of disk used, and the EC2 instance tagged with ApplicationB tracks the percentage of memory used.

Prerequisites

This post assumes that you have the following:

• An AWS account. If you don’t have an account, sign up for one at http://thinkwithwp.com.
• Amazon EC2 instances installed with AWS Systems Manager Agent (SSM Agent). If you use the Amazon Linux 2 Amazon Machine Image (AMI) from the following walkthrough, the SSM Agent is already installed. For other AMIs, refer to Manually install SSM Agent on EC2 instances for Linux.

Walkthrough

Step 1: Launch the CloudFormation template

Launch the following CloudFormation template:

The template launches the following resources:

• Two Parameter Store parameters: ApplicationAParameter and ApplicationBParameter.
• A command document called SSMDocumentInstallConfigureCWAgent.
• Two State Manager associations: ApplicationAAssociation and ApplicationBAssociation.
• An IAM role and instance profile that allows the SSM Agent and CloudWatch agent to perform actions on your behalf.

After the CloudFormation stack deploys successfully, locate the InstanceProfileName attribute in the Outputs section, and note its value.

Step 2: Launch and configure Amazon EC2 instances with tags

1. Launch two Amazon Linux 2 instances. For more information, refer to Tutorial: Get started with Amazon EC2 Linux instances.
2. For the IAM role, choose the IAM role that you deployed with the CloudFormation template.
3. For the first Amazon Linux 2 instance, configure it with the following tags:
   • Application: ApplicationA
   • Name: ApplicationA
4. For the second Amazon Linux 2 instance, configure it with the following tags:
   • Application: ApplicationB
   • Name: ApplicationB

Step 3: Verify that the instances are associated with State Manager

1. In the AWS Systems Manager console, choose Node management > State Manager.
2. Locate the two EC2 instances, and confirm that each row displays Success:1 in the Resource status count column.

Step 4: Verify that AWS Systems Manager records custom CloudWatch metrics

1. In the CloudWatch console, choose Metrics > All metrics.
2. Under Custom namespaces, confirm that the namespaces for ApplicationA and ApplicationB are created. If not, check back after about five minutes.
3. Choose ApplicationA, and confirm that the disk_used_percent metric is being collected in the InstanceId, device, fstype, and path columns.
4. Navigate back to the Custom namespaces page.
5. Choose ApplicationB, and confirm that the mem_used_percent metric is being collected for the InstanceId column.

Cleanup

To remove the resources that are no longer needed, complete the following steps:

1. Delete the CloudFormation stack. For instructions, refer to Deleting a stack on the AWS CloudFormation console.
2. Complete one of the following:
   • If you no longer need the EC2 instances, delete them. For instructions, refer to Terminate an instance.
   • If you prefer to keep the EC2 instances but remove the CloudWatch agent, sign in to the AWS Management Console, and follow the recommendations in the AWS discussion forum.

Conclusion

In this post, we explained how to use resources in AWS Systems Manager to automate the installation and configuration of CloudWatch agent for new Amazon EC2 instances for collecting custom metrics.

We encourage you to experiment with different tags and variations of the CloudWatch JSON configuration. Explore the list of metrics that the CloudWatch agent can collect, and check out this GitHub repository for various CloudFormation templates related to configuring CloudWatch. Let us know your experience in the comments, and have fun!

Here are a couple of other resources to help you get started:

• Collect metrics using the CloudWatch agent
• AWS Systems Manager State Manager

About the authors