AWS Cloud Operations Blog

Estimating AWS Config recorder costs and usage using AWS CloudTrail

AWS Config is a service that tracks configuration changes of AWS resources in your AWS account.  AWS Config uses the configuration recorder to create a configuration item whenever it detects a change to a resource type that it is recording. For example, if AWS Config is recording Amazon S3 buckets, AWS Config creates a configuration item whenever a bucket is created, updated, or deleted. A part of AWS Config pricing is based on the number of configuration items recorded in your account.  However, configuration items are dependent on workload behavior which can make it difficult to estimate the number of configuration items.

In this post, we will show how you can use AWS CloudTrail to help get an estimate of the number of configuration items if AWS Config is enabled.  AWS CloudTrail is a service that monitors and records account activity across your AWS infrastructure. These recorded events will help us determine an estimate of configuration items that could get recorded by AWS Config.  If you are utilizing AWS Config rules or AWS Config conformance packs, there will be additional charges. Please see the AWS Config pricing page for examples.

Step 1: Querying AWS CloudTrail

Prerequisites

The sample CloudTrail queries in this blog require customers have either of the following setup options:

  1. CloudTrail Lake event data store setup to receive CloudTrail Events.
  2. Amazon Athena setup to query your CloudTrail logs.

The option you choose will depend upon which of these options you have configured in your environment

Option 1: Running the query in AWS CloudTrail Lake

The following sample CloudTrail Lake query will return the total captured API events related to a list of AWS services within a given time period.  You can optionally add or remove additional AWS services to the query by updating the eventSource section of the query. The API events queried are events that could trigger a resource change in AWS Config.  There are some resource types that are not captured by the sample query that AWS Config records such as the AWS::Config::ResourceCompliance. This query will give you a starting point as to how many configuration items could get recorded if the AWS Config recorder was turned on.  The AWS Calculator can help you create an estimate for AWS Config. Please note running this query will incur an additional cost, please review the CloudTrail pricing page for the updated pricing.

  1. Navigate to the CloudTrail console .
  2. In the left-hand navigation menu, choose Lake.
  3. Choose the Editor
  4. Copy the below query and paste it into your editor window. (Note: you must replace $EDS_ID with the id of your event data store. Also, you can add additional eventSource you would like to track)
SELECT
    recipientAccountId, awsRegion, eventSource, count(* 
    ) as TotalPossibleCI 
FROM
    $EDS_ID
Where
    (eventSource like 'eks%' 
        or eventSource like 'ec2%' 
        or eventSource like 'vpc%'
        or eventSource like 'ecs%' 
        or eventSource like 'iam%' 
        or eventSource like 'autoscaling%' 
        or eventSource like 's3%' 
        or eventSource like 'rds%' 
        or eventSource like 'backup%' 
        or eventSource like 'athena%' 
        or eventSource like 'cloudtrail%' 
        or eventSource like 'cloudfront%' 
        or eventSource like 'cloudformation%' 
        or eventSource like 'code%' 
        or eventSource like 'ecr%' 
        or eventSource like 'lambda%' 
        or eventSource like 'efs%' 
    ) 
    and readOnly=False 
    and managementEvent=True 
    and eventTime > '2023-04-01 00:00:00' 
    AND eventTime < '2023-04-30 00:00:00' 
group
    by recipientAccountId, awsRegion, eventSource Order by recipientAccountId desc, TotalPossibleCI desc
  1. Next, you must replace the time range that will be searched with the time range you want to use. This will help to reduce the amount of data scanned within the query to only search the time and date specified. The date string specified after eventTime > is the earliest event timestamp that will be included, while the date string specified after eventTime < is the latest event timestamp that will be included.
  2. Click Run and then your results will show under Query Results. Please note the value of the TotalPossibleCI column will be used when estimating the AWS Config cost for the total amount of configuration items recorded within a month.
Query results screen displaying the output of the sql query.

Figure 1: AWS CloudTrail Lake query results

  1. You can then save the query run again at future time, by clicking the Save
  2. Give it the name Estimate Amount of AWS Config Configuration Items.
  3. Enter a description of Estimate of Amount of AWS Config Configuration Items.
  4. Click Save query.

Option 2: AWS CloudTrail query for Amazon Athena

To run a similar query if you have trails for CloudTrail setup to use Amazon Athena.  You can use the below sample Athena query to give you an estimate of how many potential configuration items could be created if AWS Config would be enabled.  Similar to the CloudTrail Lake query above, you can optionally add or remove additional AWS services to the query by updating the eventSource section of the query. The results of the query will give you a starting point that can be used with the AWS Calculator to estimate AWS Config cost for the amount of configuration items that could get recorded if the AWS Config recorder was turned on.  Also, please note that running this query will incur an additional cost, please review the Athena pricing page for the updated pricing.

(Note: you must replace ${tableName} with the Athena table name for CloudTrail and adjust the eventTime fields with the date range you would like to query. Also, you can add additional eventSource you would like to track)

SELECT
  recipientAccountId, awsRegion, eventSource, count(*) as TotalPossibleCI
FROM
  ${tableName}
WHERE
  (eventSource like ‘eks%’
    or eventSource like ‘ec2%’
    or eventSource like ‘vpc%’
    or eventSource like ‘ecs%’
    or eventSource like ‘iam%’
    or eventSource like ‘autoscaling%’
   or eventSource like ‘s3%’
   or eventSource like ‘rds%’
   or eventSource like ‘backup%’
   or eventSource like ‘athena%’
   or eventSource like ‘cloudtrail%’
   or eventSource like ‘cloudfront%’
   or eventSource like ‘cloudformation%’
   or eventSource like ‘code%’
   or eventSource like ‘ecr%’
   or eventSource like ‘lambda%’
   or eventSource like ‘efs%’
  )
  AND readonly = ‘false’
  AND managementEvent = ‘True’
  AND eventTime > ‘2023-04-01 00:00:00’
  AND eventTime < ‘2023-04-30 00:00:00’
GROUP BY recipientAccountId, awsRegion, eventSource
ORDER BY recipientAccountId desc, TotalPossibleCI desc;

Step 2: Create Estimate with AWS Pricing Calculator

We can use AWS Pricing Calculator to create an estimate of the monthly AWS Config cost for the amount of possible configuration items recorded if the AWS Config recorder was turned on.  Please note, that configurations items recorded are just one part of the AWS Config pricing, you would also need to estimate the amount of AWS Config rules and conformance pack evaluation that will be ran in your account.

  1. Navigate to AWS Pricing Calculator page and choose Create an Estimate.
  2. In the Find Service search box under Select Service, search for “AWS Config” and under AWS Config, choose Configure.
  3. In the Configure AWS Config section, type a description and choose the region you would like to create the pricing estimate for.
  4. In the Configuration items recorded section, under the Number of Configuration items recorded, enter in the total value of “TotalPossibleCI” from the query you ran earlier in CloudTrail.
  5. This will then give you an estimate of the AWS Config cost for the total amount of configuration items recorded within a month.
Create an Estimate screen for configuring AWS Config values for the AWS Pricing Calculator.

Figure 2: Creating an Estimate with AWS Pricing Calculator

Cleanup

If you like to remove the query saved in CloudTrail Lake, you can do the following steps:

  1. Navigate to the CloudTrail console .
  2. In the left-hand navigation menu, choose Lake.
  3. Choose the Saved queries
  4. Select the Estimate Amount of AWS Config Configuration Items
  5. Click Delete.

Conclusion

This post demonstrates how to use CloudTrail Lake to help gather statistic related to the amount of API related events that could trigger a creation of a configuration items by AWS Config.  With CloudTrail Lake you can further use additional queries to gather deeper insight to activity being captured within your account. To learn more about CloudTrail Lake see the blog article Announcing AWS CloudTrail Lake – a managed audit and security Lake.

About the authors:

Isaiah Salinas

Isaiah Salinas is a Senior Specialist Solution Architect with the Cloud Operations Team. With over 10 years of experience working with AWS technology, Isaiah works with customers to design, implement, and support complex cloud infrastructures. He also enjoys talking with others about how to use AWS services to provide solutions to their problems.

Brad Gilomen

Brad is a Principal Product Specialist for AWS CloudOps, focused on AWS Config and CloudTrail. His experience includes leading Federal Financial Enterprise Support at AWS for three years before becoming a Product Specialist in 2020. He currently works with our customers to reduce risk while staying secure and compliant while operating in the cloud.