Tag your AWS Resources consistently with AWS Resource Explorer and AWS CloudTrail

It is often challenging to consistently apply resource tags that deliver organizational benefits such as accurate cost allocation and granular access controls. Organizations often face issues with cleaning up resources in lower environments created by developers during early stages of development and testing. Without proper tagging, it can be difficult to identify experimental resources created by developers who may have left the organization or clean up resources when a project is shut down.

Tagging Amazon Web Services (AWS) resources is often overlooked in the initial stages of workload development and customers  can struggle with enforcing such discipline among developers. Using accurate, meaningful tags consistently to your AWS resources is a best practice.

One popular method used by customers that have AWS Organizations with all features enabled is to create service control policies (SCPs) that require specific tags to be applied to all resources and use tag policies to ensure valid tags are applied to the resources. AWS Organizations offers policy-based management for multiple AWS accounts and helps centrally manage environments as you scale your AWS resources.

Alternatively, if your enterprise does not currently use AWS Organizations or needs a more flexible solution, you can use AWS Resource Explorer and AWS CloudTrail to ensure appropriate tags are applied automatically. The solution described in this blog provides an automated way for ensuring your AWS resources are tagged appropriately in an asynchronous manner. It can eventually help categorize and track AWS resources to control and monitor escalating spend.

Solution overview

The auto-tagging solution applies your organization’s required tags to newly created resources using an automated workflow. This solution leverages two core services—AWS Resource Explorer and AWS CloudTrail. It includes a rule created in Amazon CloudWatch Events to schedule the tagging process. An AWS Lambda function and Amazon DynamoDB table assist in mapping untagged resources to their corresponding CloudTrail creation event.

Pre-requisites

This solution relies on AWS Resource Explorer and AWS CloudTrail for automated tagging. Hence, these services should be enabled as a pre-requisite in your account. You will need to complete the following two actions before deploying the solution:

1. Enable Resource Explorer indexing.
2. Enable CloudTrail and create a trail if there is none.

Figure 1: Auto-tagging solution workflow

Solution Flow

Following is a high-level overview of the steps needed for implementing the solution:

1. An Amazon EventBridge scheduler rule is set to run every 30 minutes to detect any new resources created.
2. The scheduler rule will trigger a Lambda function.
3. The Lambda function will scan a mapping file stored in an Amazon Simple Storage Service (Amazon S3) bucket to retrieve records containing a list of resource types and its mapped CloudTrail creation Event Name and Event Source. For this demo, the list of resource types covers Amazon Elastic Compute Cloud (Amazon EC2), S3 bucket, Lambda, and Amazon Elastic Container Service (Amazon ECS).
4. For every resource type from the mapping file, the Lambda function will query AWS Resource Explorer for any resources of that type that doesn’t have proper tagging.
5. For every resource item identifier returned from the Resource Explorer, the Lambda function will query CloudTrail events using Resource Identifier, CloudTrail Event Source, and CloudTrail Event Name using the mapping file from step 4 to find the principal information that created the resource.
6. The Lambda function then will generate a tag list based on principal information created from the resource in the CloudTrail event and invoke the Resource Groups tagging API to auto tag the resource with the generated tag list.

Deploying the Solution

If this is your first time using AWS Cloud Development Kit (CDK), you will need to install it on your system to implement the following steps:

• Clone the application to your local machine.
  • git clone https://github.com/aws-samples/resource-autotagger.git
• Change to the project directory.
  • cd resource-autotagger
• Install node dependencies for the AWS Cloud Development Kit (AWS CDK) and Lambda functions.
  • npm install
• Execute below step to bootstrap AWS CDK if it was never set up in your AWS account.
  • cdk bootstrap
• Deploy the solution resources with AWS CDK.
  • cdk deploy

After deploying the solution, within a few minutes you will notice user identification tags such as “Created by” and “IAM Role name” are applied to the resources.

Extending the Solution for other AWS Services

This demo solution covers the auto-resource tagging of Amazon EC2, S3 bucket, Lambda and Amazon ECS based on the mapping information stored in a mapping file in an S3 bucket. To extend this solution for other AWS services, update the mapping file with additional resource types and its corresponding CloudTrail creation Event Name, Event Source and Resource Explorer resource type value. Note that AWS Identity and Access Management (IAM) permissions required to add tagging to additional resource types need to be added to the AWS Lambda execution policy.

Following is an example of steps to take to add Amazon API Gateway resource creation tagging:

1. Identify CloudTrail Event Name and CloudTrail Event Source for API Gateway REST API creation and Resource Explorer resource type for API Gateway REST API query. This information will be required for updating the mapping file in step 2 and step 3. For Amazon API Gateway, the creation event name for CloudTrail is “CreateRestApi”, the event source is “amazonaws.com” and the Resource Explorer Resource Type is “apigateway:restapis”.

2. Go to the Amazon S3 console and open the S3 bucket name starting with “resourceautotagcdkstack-resourceautotagbucket”. It contains a JSON file named “json”.

3. Update the JSON file by adding the following JSON structure with values identified in step 1 and upload to the bucket. Note, the Global property only applies to resources that are globally unique (such as, S3 bucket). Properties starting with “CT” refer to CloudTrail properties and “RE” refers to Resource Explorer properties.

{
"CTEventName": "CreateRestApi",
"CTEventSource": "apigateway.amazonaws.com",
"REResourceType": "apigateway:restapis",
"Global": false
}

4. Go to the IAM console. Click Roles in the left-hand navigation and search for the IAM role for Lambda execution created from the AWS CDK deployment, which has a name starting with ResourceAutoTagCdkStack-.

5. Click on the role detail and click Policy then inlinePolicy. In the Edit policy screen add the IAM statement as shown in the following code block to allow the Lambda function to add tags to the REST API. Replace <region> with your deployment region.

{
"Sid": "ResourceAutoTaggerAPIGatewayTagging",
"Effect": "Allow",
"Action": ["apigateway:POST","apigateway:PUT","apigateway:PATCH"],
"Resource": "arn:aws:apigateway:<region>::/*/*"
}

Figure 2: Sample Lambda function execution policy after adding the permissions.

6. Click Next and Save Changes.

Cleanup

To avoid charges after you test the auto-tagging function, delete the AWD CDK stack by running the following command from root directory of cloned repository.

cdk destroy

Considerations

AWS Resource Explorer has a limit of 10,000 search operations per month and can return a maximum of 1000 results per query. This allows for a maximum of 10,000,000 AWS resources to be tagged per account per month. Additionally, AWS resources can have a maximum of 50 tags per resource. This solution will be unable to add your specified tag to resources that already have 50 tags. The solution currently is limited to one AWS account and is not compatible to tag across multiple accounts.

Conclusion

In this blog, we demonstrated how AWS Resource Explorer and Amazon CloudTrail can be used in conjunction to automate resource tagging. This solution is an alternative method to apply required tags to AWS resources that is flexible, customizable and does not impact existing workloads. It does not rely on enabling service control policies and runs asynchronously in the background to ensure AWS resources are tagged.

Contact an AWS Representative to know how we can help accelerate your business.

Further Reading

• Using AWS Cost allocation tags
• Granular access control using Attribute-based access control (ABAC)
• Best practices for Tagging AWS Resources

About the Authors