How to use AWS Config proactive rules and AWS CloudFormation Hooks to prevent creation of noncompliant cloud resources

Balancing developer freedom and governance controls is a key challenge faced by organizations that are adopting cloud. On one hand, developers need the freedom to innovate and develop new applications and services quickly and on the other, organizations need to maintain control over the resources used and the data processed in order to ensure compliance with internal policies, regulatory standards, and best practices.

With the AWS Config service, you can assess, audit, and evaluate configurations of your Amazon Web Services (AWS) resources. Customers use AWS Config to track configuration changes made to their cloud resources and check if those resources match the desired configurations using AWS Config Rules.

Typically, customers run compliance checks using AWS Config Rules against the resources after they have been created or updated. AWS Config Rules can also run proactively to check for compliance prior to resource provisioning.

AWS CloudFormation Hooks is a feature of AWS CloudFormation that lets you run code to inspect the configuration of your AWS resources before provisioning. If non-compliant resources are found, AWS CloudFormation hook returns a failure status and either fails the operation or provides a warning and allows the operation to continue based on the hook failure mode.

Solution Overview

In this blog post we will show you how to use AWS Config proactive rules and AWS CloudFormation Hooks to evaluate your AWS resources compliance before the resources are provisioned. Resources are provisioned only if they are compliant, lowering security risks and operational overhead.

Figure 1 – Resource configuration evaluation with CloudFormation hooks and AWS Config proactive rules

In our example, once a new AWS CloudFormation stack creation is initiated (1), the hook code evaluates resource configuration against all applicable AWS Config rules (2). If the configuration is non-compliant (3) AWS CloudFormation fails the operation (4). If the configuration is compliant (5), AWS CloudFormation proceeds with resource provisioning (6).

Deployment instructions

To demonstrate this functionality, we will create two AWS Managed Config Rules:

• rds-storage-encrypted – This rule checks if storage encryption is enabled for Amazon Relational Database Service (RDS) DB instances. The rule is non-compliant if storage encryption is not enabled.
• rds-instance-public-access-check – This rule checks whether Amazon Relational Database Service (RDS) instances are not publicly accessible. The rule is non-compliant if the publiclyAccessible field is true in the instance configuration item.

Next, we will develop and test a hook that will evaluate an RDS DB instance configuration against those rules.

All required configuration files and source code are in the GitHub repository, let’s clone it:

$ mkdir project
$ cd project
$ git clone https://github.com/aws-samples/proactive-evaluation-with-cf-hooks-aws-config-blog-source.git

Create AWS Config Rules

Navigate to config-rules directory:

$ cd proactive-evaluation-with-cf-hooks-aws-config-blog-source/config-rules

Make sure AWS Command Line Interface (CLI) is installed and configured with your AWS credentials. Execute the following command to create rds-storage-encrypted rule:

$ aws configservice put-config-rule --config-rule file://rds-storage-encrypted-config-rule.json

Create rds-instance-public-access-check rule:

$ aws configservice put-config-rule --config-rule file://rds-instance-public-config-rule.json

Now that we have the rules in place, let’s build the AWS CloudFormation hook using Python.

Environment setup

Make sure your development environment has Docker and Python version 3.6 or later installed.

Navigate to the root directory, install AWS CloudFormation CLI (cfn) and AWS CloudFormation Resource Provider Python Plugin:

$ cd ../../
$ mkdir cf-hook
$ cd cf-hook
$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install cloudformation-cli cloudformation-cli-python-plugin --use-feature=2020-resolver

Initiate project

Run cfn init command. The command launches a wizard that walks you through setting up the project.

$ cfn init

Initializing new project
Do you want to develop a new resource(r) or a module(m) or a hook(h)?.
>> h

What's the name of your hook type?
(Organization::Service::Hook)
>> Demo::Testing::ConfigRuleHook

Select a language for code generation:
[1] python36
[2] python37
[3] python38
[4] python39
(enter an integer):
>> 4
Use docker for platform-independent packaging (Y/n)?
This is highly recommended unless you are experienced
with cross-platform Python packaging.
>> Y

You might get a warning message “Could not generate schema docs due to missing type info or schema”, you can ignore that.

Hook Model

Hook model is a JSON schema that defines the hook, its properties, and their attributes. The schema also contains permissions required for the hook. You need to specify permissions for each of your hook handlers. Replace the auto-generated demo-testing-configrulehook.json with the version from the GitHub repository:

$ cp ../proactive-evaluation-with-cf-hooks-aws-config-blog-source/demo-testing-configrulehook.json .

Run cfn generate to update hook-role.yaml with proper IAM permissions

$ cfn generate

Hook Handler Code

Invocation points specify the exact point where the hook is invoked. Hook handlers hosts executable custom logic for these points. For example, an invocation point of the CREATE operation uses a preCreate handler. For our use case, we will implement the preCreate and perUpdate invocation points. The code will perform resource evaluation, wait until the evaluation is completed and return either SUCCESS or FAILURE based on the evaluation results. Copy the handler.py from the GitHub repository:

$ cp ../proactive-evaluation-with-cf-hooks-aws-config-blog-source/src/handlers.py src/demo_testing_configrulehook/

Register and enable the hook

To register the hook run the following command from the cf-hook root directory:

$ cfn submit --set-default

The command should return the result:

Starting Docker build. This may take several minutes if the image 'public.ecr.aws/sam/build-python3.9' needs to be pulled first.
Successfully submitted type. Waiting for registration with token '9b3b0e4d-78d8-448d-8102-e6b0d8280c37' to complete.
Registration complete.
{
  'ProgressStatus': 'COMPLETE',
  'Description': 'Deployment is currently in DEPLOY_STAGE of status COMPLETED',
  'TypeArn': 'arn:aws:cloudformation:us-east-1:ACCOUNT_ID:type/hook/Demo-Testing-ConfigRuleHook',
  'TypeVersionArn': 'arn:aws:cloudformation:us-east-1:ACCOUNT_ID:type/hook/Demo-Testing-ConfigRuleHook/00000001',
  'ResponseMetadata': {
    'RequestId': '107d07fd-f5e3-41ce-b2dc-a414c96d46f6',
    'HTTPStatusCode': 200,
    'HTTPHeaders': {
      'x-amzn-requestid': '107d07fd-f5e3-41ce-b2dc-a414c96d46f6',
      'date': 'Thu, 09 Mar 2023 16:04:22 GMT',
      'content-type': 'text/xml',
      'content-length': '683'
    },
  'RetryAttempts': 0
  }
}

Use the list-types command to list your newly registered hook to get its ARN:

$ aws cloudformation list-types

The command returns the following output:

{
  "TypeSummaries": [
    {
      "Type": "HOOK",
      "TypeName": "Demo::Testing::ConfigRuleHook",
      "DefaultVersionId": "00000001",
      "TypeArn": "arn:aws:cloudformation:us-east-1:ACCOUNT_ID/type/hook/Demo-Testing-ConfigRuleHook",
      "LastUpdated": "2021-08-04T23:00:03.058000+00:00",
      "Description": "Runs AWS config rules evaluation for supported cloud resources before provisioning"
    }
  ]
}

Capture the TypeArn from the output and enable the hook:

$ export HOOK_TYPE_ARN=<hook-arn>
$ aws cloudformation set-type-configuration  --configuration "{\"CloudFormationConfiguration\":{\"HookConfiguration\":{\"TargetStacks\":\"ALL\",\"FailureMode\":\"FAIL\"}}}" --type-arn $HOOK_TYPE_ARN

Testing the hook

To test the hook, we will try to create an RDS instance. Navigate to the templates directory:

$ cd ../proactive-evaluation-with-cf-hooks-aws-config-blog-source/templates

The directory contains a sample AWS CloudFormation template (rds.yaml) that we can use for testing. Open rds.yaml in your favorite editor and notice that the StorageEncrypted property is set to false. This makes the resource non-compliant as it does not meet the rds-storage-encrypted rule requirements. Let’s see this in action, create a stack with the following command (replace <username>, <password> and <security group id> with actual values):

$ aws cloudformation create-stack --stack-name non-compliant-rds-stack --template-body file://rds.yaml --parameters ParameterKey=DBUsername,ParameterValue=<username> ParameterKey=DBPassword,ParameterValue=<password> ParameterKey=SecurityGroupID,ParameterValue=<security group id>

View stack events:

$ aws cloudformation describe-stack-events --stack-name non-compliant-rds-stack

We can see the stack failed to create since our database configuration is non-compliant:

{
  "StackId": "arn:aws:cloudformation:us-east-1:ACCOUNT_ID:stack/non-compliant-rds-stack/0cdba710-d956-11ed-8a81-0eaeb97704dd",
  "EventId": "DBInstance-b5b50fe4-cb80-4610-84e9-351d98f330c3",
  "StackName": "non-compliant-rds-stack",
  "LogicalResourceId": "DBInstance",
  "PhysicalResourceId": "",
  "ResourceType": "AWS::RDS::DBInstance",
  "Timestamp": "2023-04-12T17:18:40.388000+00:00",
  "ResourceStatus": "CREATE_IN_PROGRESS",
  "HookType": "Demo::Testing::ConfigRuleHook",
  "HookStatus": "HOOK_COMPLETE_FAILED",
  "HookStatusReason": "Hook failed with message: Resource is NON-COMPLIANT",
  "HookInvocationPoint": "PRE_PROVISION",
  "HookFailureMode": "FAIL"
}

You can also review the hook code execution logs in CloudWatch (demo-testing-configrulehook-logs log group):

{
  'ResourceEvaluationId': '2b32462d-be4e-48bb-9112-725b73c9a293-1877679fdd3',
  'EvaluationMode': 'PROACTIVE',
  'EvaluationStatus': {
    'Status': 'SUCCEEDED'
  },
  'EvaluationStartTimestamp': datetime.datetime(2023, 4, 12, 17, 18, 39, 101000, tzinfo = tzlocal()),
  'Compliance': 'NON_COMPLIANT',
…
}

Delete the stack:

$ aws cloudformation delete-stack --stack-name non-compliant-rds-stack

Open rds.yaml file and change StorageEncrypted property to true and deploy again (replace <username>, <password> and <security group id> with actual values):

$ aws cloudformation create-stack --stack-name compliant-rds-stack --template-body file://rds.yaml --parameters ParameterKey=DBUsername,ParameterValue=<username> ParameterKey=DBPassword,ParameterValue=<password> ParameterKey=SecurityGroupID,ParameterValue=<security group id>

List stack events:

$ aws cloudformation describe-stack-events --stack-name compliant-rds-stack

This time we can observe the resource is compliant:

{
  "StackId": "arn:aws:cloudformation:us-east-1:291701893262:stack/compliant-rds-stack/bee5cee0-d956-11ed-ba8c-0ef5781b88cf",
  "EventId": "DBInstance-f3e4fa84-0e5e-4255-a441-a518093cefee",
  "StackName": "compliant-rds-stack",
  "LogicalResourceId": "DBInstance",
  "PhysicalResourceId": "",
  "ResourceType": "AWS::RDS::DBInstance",
  "Timestamp": "2023-04-12T17:23:37.384000+00:00",
  "ResourceStatus": "CREATE_IN_PROGRESS",
  "ResourceStatusReason": "Hook invocations complete.  Resource creation initiated"
},
{
  "StackId": "arn:aws:cloudformation:us-east-1:ACCOUNT_ID:stack/compliant-rds-stack/bee5cee0-d956-11ed-ba8c-0ef5781b88cf",
  "EventId": "DBInstance-d1c58f4b-0992-44f7-aa20-5ef356d9cf1d",
  "StackName": "compliant-rds-stack",
  "LogicalResourceId": "DBInstance",
  "PhysicalResourceId": "",
  "ResourceType": "AWS::RDS::DBInstance",
  "Timestamp": "2023-04-12T17:23:37.246000+00:00",
  "ResourceStatus": "CREATE_IN_PROGRESS",
  "HookType": "Demo::Testing::ConfigRuleHook",
  "HookStatus": "HOOK_COMPLETE_SUCCEEDED",
  "HookStatusReason": "Hook succeeded with message: Successfully invoked HookHandler for target AWS::RDS::DBInstance. Resource is COMPLIANT",
  "HookInvocationPoint": "PRE_PROVISION",
  "HookFailureMode": "FAIL"
}

Our hook is configured to process create and update operations so if you update an existing stack with supported resources, the same hook code will be executed, and if the submitted configuration is not compliant, the update operation will fail.

Delete the stack:

$ aws cloudformation delete-stack --stack-name compliant-rds-stack

Cleanup

To delete the resources created in this blog post run the following commands:

Delete AWS Managed Config Rules:

$ aws configservice delete-config-rule --config-rule-name rds-storage-encrypted
$ aws configservice delete-config-rule --config-rule-name rds-instance-public-access-check

Disable the hook

$ aws cloudformation set-type-configuration --configuration "{\"CloudFormationConfiguration\":{\"HookConfiguration\":{\"TargetStacks\":\"NONE\",\"FailureMode\":\"FAIL\"}}}" --type-arn $HOOK_TYPE_ARN

Deregister the hook

You might need to deregister non-default versions before attempting to deregister the type.

Get a list of all versions:

$ aws cloudformation list-type-versions --type HOOK --type-name Demo::Testing::ConfigRuleHook

Delete all non-default versions:

$ aws cloudformation deregister-type --type HOOK  --type-name Demo::Testing::ConfigRuleHook --version-id <non default version you want to delete>

Deregister the type:

$ aws cloudformation deregister-type --type HOOK --type-name Demo::Testing::ConfigRuleHook

Delete stacks

Delete the CloudFormation stack demo-testing-confirulehook-role-stack. This stack contains the IAM role for the hook:

$ aws cloudformation update-termination-protection --stack-name demo-testing-configrulehook-role-stack --no-enable-termination-protection
$ aws cloudformation delete-stack --stack-name demo-testing-configrulehook-role-stack

Delete the CloudFormation stack CloudFormationManagedUploadInfrastructure. This stack contains the required infrastructure for CloudFormation hook such as Amazon S3 buckets, IAM roles, KMS keys, etc. Prior to deleting the stack, make sure that the buckets are empty and TerminationProtection is disabled:

$ aws cloudformation delete-stack --stack-name CloudFormationManagedUploadInfrastructure

Conclusion

In this post, we demonstrated how you can use AWS Config proactive rules and AWS CloudFormation Hooks to evaluate configuration of AWS resources. The proactive evaluation of AWS resources is a preventative measure that helps you maintain compliance, lower your security risk and reduce operational overhead by identifying non-compliant resources before they are provisioned.

Eugene Kim Eugene is a Sr. Solutions Architect at Amazon Web Services in New York, brings over 15 years of experience in designing and implementing scalable and complex solutions in AWS. He specializes in container and serverless technologies.

Ram Konchada Ram is a Senior AWS Solutions Architect in New York City. He works with enterprise customers, enabling them with digital transformation initiatives leveraging AWS and other technologies. Ram loves watching sports and is a tennis enthusiast.

Bappaditya Datta Bappaditya Datta is a Sr. Solution Architect in AWS North America focusing on data & analytics. He is helping customers across different industries to design and build secure, scalable, and highly available solutions, addressing their business needs and bringing innovations. Prior to AWS, Bappaditya worked as Technical Architect helping pharmaceutical customers adopt AWS cloud for their data & analytics needs.