How to Track Changes to Auto Scaling Groups Using AWS Config

Recently, AWS Config announced support for Auto Scaling groups. You can now track configuration changes in Auto Scaling groups, such as minimum, maximum, and desired capacities, termination policies, scaling policies, subnets, and instance protection settings.

You can also use a new managed AWS Config rule to check whether the Auto Scaling groups associated with your load balancers are using the Elastic Load Balancing health checks to determine the health state of your Amazon EC2 instances.

In this post, I describe two possible ways that you could track changes to Auto Scaling groups and evaluate compliance. I’ll walk you through each scenario.

AWS services
AWS Config enables you to assess, audit, and evaluate the configurations of your AWS resources. It continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations. With AWS Config, you can review changes in configurations and relationships between AWS resources, dive into detailed resource configuration histories, and determine your overall compliance against the configurations specified in your internal guidelines.

An Auto Scaling group is a collection of EC2 instances, with similar configuration and characteristics, that are used for the purpose of scaling and management. These EC2 instances in the Auto Scaling group are treated as a logical group. You can configure the Auto Scaling group with the capacity that you want for instances required for the application to operate. You can configure the group with the minimum and maximum instances, depending on which instances Auto Scaling can launch or terminate based on dynamic scaling.

Example: Track configuration changes

Let’s consider a scenario where a customer notices an increase in his Amazon Elastic Compute Cloud (Amazon EC2) bill for a prior month. He notices that there are Amazon EC2 charges related to the use of an m4.xlarge instance type. The customer is confused because his applications have always used t2.micro instances, so he investigates further. He notices that when a scale out happens, the instances launched by the Auto Scaling group are of type m4.xlarge, instead of t2.micro. So it appears that the Auto Scaling launch configuration must have been modified.

AWS Config can help you investigate in this scenario and determine if and when any configuration changes occurred on the Auto Scaling launch configuration, since it maintains a history of all configuration changes to the Auto Scaling group. We use the AWS Management Console in this scenario, but you can use the AWS CLI or SDKs as well.

First, log in to the AWS Config console. For Resources, choose AutoScaling: AutoScaling Group. Choose Look up to see the list of Auto Scaling groups that AWS Config has automatically discovered in your account. The following screenshot shows an example Auto Scaling group on the Resource inventory page.

Look at the Config timeline for the AS- APP1 – AS group. The following screenshot shows the timeline and configuration details.

The timeline shows you all configuration changes that were made after the Auto Scaling group was created.

Choose Changes and notice that a change was made on the 31st August to the Auto Scaling group configuration, when the launch configuration was changed from AS-APP1-Launch config to AS-APPv2-Launch config.

Let’s open the EC2 console to check the Auto Scaling launch configurations. We can confirm that the instance type associated with the new launch configuration is m4.xlarge.

Here is another example of tracking configuration changes using AWS Config. In this example, let’s assume that you are seeing performance degradations for applications hosted on your EC2 instances. Open the Amazon CloudWatch console. You can see spikes but are unable to understand why Auto Scaling did not dynamically scale out based on the CPU threshold scaling policy which was set to scale at 60 percent of average CPU.

Go back to the AWS Config console. For Resources, choose AutoScaling: ScalingPolicy. Choose Look up to see the list of Auto Scaling Policies that AWS Config has automatically discovered in your account. The following screenshot shows an example Auto Scaling policy on the Resource inventory page

Look at the Config timeline for Scale Group Size scaling policy. The following screenshot shows the timeline and configuration details.

Choose Changes and notice that a change was made on 4th September to the Scaling policy configuration, when the target value for Average CPU threshold was changed from 60 to 90 percent. This explains why the scale out did not happen as soon as the CPU spiked over 60 percent.

Using this information, you can fix these Auto Scaling configuration issues in the EC2 console.

Another common scenario some customers have experienced is noticing too many instances in their account. This is because the instance termination policy has been enabled, which does not terminate your instances when the scale-in happens. This can be identified using the AWS Config service as well.

Example: Verify Heath Check using AWS Config rules

In addition to tracking the configuration of your Auto Scaling service, you can use AWS Config rules to check your Auto Scaling groups against best practices and internal policies.

For example, use the prebuilt rules called autoscaling-group-elb-healthcheck-required to check whether your Auto Scaling groups that are associated with a load balancer are using Elastic Load Balancing health checks.

Auto Scaling groups use Amazon EC2 health checks by default to determine the health of an instance. In situations where EC2 instances are running behind a load balancer, it’s beneficial to leverage the load balancer health checks for EC2 instances instead of Amazon EC2 health checks. This is because the load balancer health checks can determine the health of the application running on top of the EC2 instances, unlike the Amazon EC2 health checks.

Here’s an example that shows how you can use AWS Config to troubleshoot a scenario where customers notice service errors or failures for the applications hosted on Amazon EC2 behind a load balancer even though the EC2 instance is healthy, reachable, and running.

On the EC2 console, you can see that the EC2 instances healthy and running. But customers are still reporting errors while running your application. After some troubleshooting you determine that there are issues with the web service running on top of your EC2 instances. Elastic Load Balancing (ELB) load balancers have the capability to detect such failures via application-specific health checks and stop directing traffic to the affected EC2 instances. But since the Auto Scaling group isn’t configured to leverage the ELB health checks, customer requests are still hitting the unhealthy EC2 instances.

AWS Config can help you identify Auto Scaling groups with this misconfiguration so that you can take corrective action. With the prebuilt rules called autoscaling-group-elb-healthcheck-required we can identify Auto Scaling groups that do not use ELB health checks. These resources are “non-compliant” as far as the AWS Config rule is concerned, as seen in the AWS Config console:

Go to the EC2 console to change the health check type to ELB in the Auto Scaling group. When you return to the AWS Config console, you can see that the compliance status for the Auto Scaling group changes to the compliant state.

Summary
With support for Auto Scaling in AWS Config, you can track configuration changes to your Auto Scaling groups and associated resource types. You can use the pre-built AWS Config rule or you can create custom rules to scan the configuration of these resources and verify compliance with best practices, internal guidelines, and regulatory requirements.

About the Authors

Varun Pole is a Solutions Architect at Amazon Web Services. He enjoys working with customers providing technical guidance and assist in architect and build solutions and help them make the best use of AWS services. In his free time, he enjoys traveling, cooking and photography.

Sid Gupta is a Sr. Product Manager for AWS Config. AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. Sid enjoys working with customers and help them implement cloud security best practices. In his spare time, he enjoys hiking, reading and spending time with his kids.