AWS Cloud Operations Blog

Creating event-based notifications and remediation in AWS Service Catalog using AWS Config

AWS Service Catalog allows organizations to quickly let their users deploy approved IT services to organize, govern, and provision cloud resources on AWS. However, users launching multiple instances can cause issues because:

  • Some instances are not always active.
  • The lack of limits on the number of active instances can result in an organization’s costs going over budget.

AWS Service Catalog customers asked the AWS Cloud Center of Excellence (CCoE) team for help with governing this resource usage. Organizations asked for the ability to trigger actions under distinct criteria so they can:

  • Stay within their budgets.
  • Identify bottlenecks.
  • Fine tune the products they use in AWS Service Catalog.

AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. Using AWS Config, you can record configuration changes to AWS Service Catalog portfolios, products, and provisioned products. You can also create rules to run AWS Lambda functions to trigger custom actions as these events occur.

In this post, I show how to use AWS Config to enable notifications when an AWS Service Catalog product launches. I also show how to trigger corrective actions when users launch products beyond the threshold that their organization sets.

Background

Here’s a quick review of some terms used in this post:

  • A product is an IT service that you want to make available for deployment on AWS. It consists of one or more AWS resources.
  • A portfolio is a collection of products, together with configuration information.
  • A provisioned product is a stack. When an end user launches a product, the instance of the product that is provisioned by AWS Service Catalog is a stack with the resources necessary to run the product.
  • AWS resources are entities that you create and manage with the AWS Management Console, the AWS Command Line Interface (AWS CLI), or the AWS SDKs.
  • A configuration item represents a point-in-time view of the various attributes of a supported AWS resource that exists in your account.
  • With AWS Config, you can create custom rules. You set up a trigger when configuration changes happen on monitored resources.

For more information on AWS Service Catalog, see the post on how to Control AWS resources available to your users using AWS Service Catalog.

Overview

This post refers to two personas:

  • Administrator The AWS Service Catalog administrator, who is responsible for developing, building, and maintaining the AWS Service Catalog portfolios and products.
  • End user A user or group (known as the “end user” in this post) who consumes an AWS Service Catalog product from a list of products that the administrator approved.

For this exercise, the administrator should have an AdministratorAccess­­­ policy attached, and the end user should have access to the AWSServiceCatalogEndUserAccess managed policy.

As the administrator, you perform the following tasks:

  •  Deploy an AWS CloudFormation stack that creates Lambda functions.
  • Deploy AWS Config rules to generate events when any changes occur to AWS Service Catalog products. The AWS Config rules invoke the respective Lambda functions when an event is generated. (You learn more about the functionality of these Lambda functions later.)
  • Deploy a second AWS CloudFormation stack to:
    • Create an AWS Service Catalog portfolio.
    • Associate a product.
    • Enable launch constraint.
    • Grant access to the end user.

As an end user, you provision the product and validate the functionality at various scenarios. For example, you could provision a product beyond the threshold or delete a product.

How it works

The following diagram shows the list of actions that happen on a product create, update, or delete.

  1. A product is associated with a portfolio.
  2. With that association, the AWS Config rule ScProductConfigRule-<region> triggers an event due to a configuration change to the resource type ServiceCatalog:CloudFormationProduct.
  3. The event generated by AWS Config invokes a Lambda function.
  4. Lambda updates the Parameter Store and logs the event.
  • a) The event generated by AWS Config invokes a Lambda function that creates the following parameters with default values in the AWS Systems Manager Parameter Store:
    • /ServiceCatalog/<portfolio-id>/<product-id>/ACTION (default: NOTIFY)
    • /ServiceCatalog/<portfolio-id>/<product-id>/CURRENT (default: 0)
    • /ServiceCatalog/<portfolio-id>/<product-id>/MAXVALUE (default: 5)
  • b) The Lambda function adds an Amazon CloudWatch log when an event is triggered due to addition, removal, activation, or deactivation of a product version. (Enabling notifications for these events is not within the scope of this blog post.)

The following diagram shows the list of actions that happen when a product is provisioned or deleted.

 

  1. An end user provisions a product from their permitted list of products.
  2. The triggered AWS Config rule ScProvisionedProductConfigRule-<region> triggers an event due to a configuration change to the resource type ServiceCatalog:CloudFormationProvisionedProduct.
  3. The event generated by AWS Config invokes a Lambda function.
  4. The Lambda function does the following when a product is disassociated from the portfolio.
    • Checks for the provisioned products for that portfolio/product.
    • If no provisioned products are left out, removes the associated parameters from the Parameter Store.
  5. The Lambda function triggers a corrective action when an AWS Config event is received. Following actions are taken, depending on the number of instances provisioned and the value of the ACTION key.
    • a) Increments the associated parameter CURRENT. On reaching the threshold, takes ACTION as specified:
      • If ACTION = NOTIFY (default): Generates an email notification.
      • If ACTION = TERMINATE: Auto terminates the provisioned product.
      • If ACTION = DISASSOCIATE:
        • The product is disassociated when CURRENT=MAXVALUE.
        • The product is re-associated when CURRENT < MAXVALUE.
    • b) When a provisioned product is terminated:
      • Reduce the value of the associated parameter’s CURRENT count value.
      • If provisioned products exist, leave the parameters as is. They are rechecked if the product is associated with the portfolio in the future.

Prerequisites

Make sure to enable AWS Config before continuing. This service is regional and is not enabled by default. For help with setup, see Setting Up AWS Config with the Console and Getting Started with AWS Config.

Creating event-based notifications and remediation in AWS Service Catalog

Try the following scenario to verify the flow. To keep this process simple, perform all these operations in the US West (Oregon) Region. To try it out in other Regions, replace the Region name where applicable.

Step 1: Obtain the setup material

  1. Download the createScConfigRules.zip file to a local directory.
  2. Log in to your Amazon S3 console as an administrator and choose Create bucket.
  3. On the Create bucket screen, for Bucket name, enter awsconfigsc-<randomstr>. Note the bucket name, as you use it later.
  4. Select US West (Oregon) as the Region and choose Next.
  5. Use the default values in remaining screens, choose Next, Next, and Create bucket.
  6. Upload the createScConfigRules.zip file to the newly-created bucket.

Step 2: Launch the AWS CloudFormation stack

  1. Select this launch link to launch the AWS CloudFormation stack in the US West (Oregon).
  2. On the Select Template screen, choose Next.
  3. On the Specify Details screen, for SourceBucket, enter the value awsconfigsc-<randomstr> that you noted in Step 1.2.a. Enter a desired email address under EmailNotification and choose Next. Refer to the following screenshot.
  4. On the Options screen, choose Next.
  5. On the Review screen, scroll down and select the I acknowledge that AWS CloudFormation might create IAM resources check box, and then choose Create.
  6. While this stack is in progress, the email-id specified under EmailNotification (for this exercise, it is scadmin@example.com) receives a notification.
  7. After receiving the email, choose Confirm Subscription in the email.
  8. For Status, wait for the value to change to CREATE_COMPLETE before proceeding further.

Upon successful execution of the stack, AWS CloudFormation creates the following resources:

  • ScProductConfigRule-<region>: AWS Config rule that triggers an event due to configuration change to resource type ServiceCatalog:CloudFormationProduct
  • ScProvisionedProductConfigRule-<region>: AWS Config rule that triggers an event due to configuration change to resource type ServiceCatalog:CloudFormationProvisionedProduct
  • createScProductLambda-<stack-name>: Lambda function triggered by AWS Config rule ScProductConfigRule-<region>
  • createScProvisionedProductLambda-<stack-name>: Lambda function triggered by AWS Config rule ScProvisionedProductConfigRule-<region>
  • AWSConfigSC-EmailNotificationTopic-<rand-str>: SNS notification topic

Step 3: Configure AWS Service Catalog

  1. Select this AWS CloudFormation launch link to launch the AWS CloudFormation stack in the US West (Oregon).
  2. On the Select Template screen, choose Next.
  3. On the Specify Details screen, leave the default Stack Name and choose Next.
  4. On the Options screen, choose Next.
  5. On the Review screen, scroll down and select the I acknowledge that AWS CloudFormation might create IAM resources check box, and then choose Create.

Upon successful execution of the stack, AWS CloudFormation does the following:

  • Creates an AWS Service Catalog portfolio Sample Portfolio for SC-AWS Config Blog.
  • Creates an AWS Service Catalog product Amazon S3 Private Encrypted Bucket.
  • Creates an IAM user, group, and a launch constraint role.
  • Associates the product with the newly-created portfolio.
    • Configures the user access and launch constraints for the AWS Service Catalog portfolio.

6. In the Outputs section of the AWS CloudFormation stack, do the following:

  • Note the value of ServiceCatalogEndUserArn. The user name is after the last backslash of Arn. You’ll use the user name to log in as the AWS Service Catalog end user. The Arn is used as the input parameter while launching the product.
  • Select the value of ServiceCatalogEndUserPassword, and choose Retrieve secret value to retrieve the temporary password for the ServiceCatalogEndUser. Note down the password, as you will use it in Step 5.2.

Step 4: Verify the SSM parameters

  1. Access the SSM Parameter Store. Verify the default values set when an AWS Service Catalog product is associated with a portfolio as part of Step 3.1.
  2. Select the parameter that ends with /CURRENT (must be 0 for a new product/portfolio).
  3. Select the parameter that ends with /MAXVALUE (the default parameter is 2).
  4. Select the parameter that ends with /ACTION (the default is NOTIFY).

You could edit the ACTION and MAXVALUE parameters to configure other supported behaviors. However, for this exercise, leave them as defaults.

Step 5: Provision an AWS Service Catalog product

  1. If you are still logged in as an administrator, log out from the current console session. Log back in to AWS Management Console using the user name and password of the end user you noted in Step 3.6a. Change the password as prompted.
  2. Search for catalog and choose Service Catalog.
  3. On the Products list screen to the left of Amazon S3 Private Encrypted Bucket, select the three vertical dots icon and choose Launch product.­­
  4. On the Product Version screen, for Name, enter a name of your choice . The latest version is already selected. Choose NEXT.
  5. In the Parameters screen, enter as the value for S3UserARN the ServiceCatalogEndUserArn you noted in Step 3.6a.
  6. On the TagOptions screen, provide the TagOptions as needed and choose NEXT.
  7. On the Notifications screen, leave the default settings and choose NEXT.
  8. On the Review screen, review the options selected, and choose Launch.

Test user constraints

  1. To test your constraints as a user, repeat steps 5.3 to 5.8, and launch the same product again. The registered email-id receives a LIMIT_REACHED email notification.
  2. Launch one more instance of the same product using steps 5.3 to 5.8 again. After launching the third product, the registered email-id receives a LIMIT_EXCEEDED email. Remember that the default MAXLIMIT value was set to 2.

By default, the ACTION is set to NOTIFY. Other supported actions like TERMINATE and DISASSOCIATE are described in the additional exercises later in this post.

Step 6: Delete a provisioned product

Here is how to delete the provisioned product while you are still logged in as an end user in AWS Service Catalog.

  1. In the AWS Service Catalog Console left navigation pane, choose Provisioned products.
  2. To the left of any of the products you just created, select the three vertical dots icon and choose Terminate provisioned product. The registered email-id receives a LIMIT_REACHED email notification.
  3. Delete all the other provisioned products.

Step 7: Disassociate a product from a portfolio

  1. After you delete all the provisioned products, log out as the end user and log back in to the AWS Management Console as an administrator.
  2. In the AWS AWS CloudFormation console, under Stack Name, select the check box to the left of the SCSetup stack. Choose Actions, Delete Stack. This action deletes the AWS Service Catalog product, portfolio, user name, group, and launch constraint role that you created for this exercise.

The Parameter Store values clear as the product is disassociated from the portfolio.

Additional scenarios

For additional practice, try the following exercises.

Modify the default settings by setting ACTION to TERMINATE

  1. Log in as administrator and repeat Step 1 to Step 4.
  2. On the Systems Manager Parameter Store dashboard:
    • Select the parameter that ends with /ACTION.
    • Choose Edit on top right side of the pane, change the value from NOTIFY to TERMINATE, and then choose Save Changes.
  3. Log out of the AWS Management Console as administrator.
  4. Follow the steps in Step 5.3 to 5.9 and launch the AWS Service Catalog product Amazon S3 Private Encrypted Bucket as an end user.
  5. Launch the same AWS Service Catalog product a second time. The registered email-id receives a LIMIT_REACHED notification.
  6. Launch the same AWS Service Catalog product for the third time. The product launches and is automatically terminated as it exceeds the MAXVALUE value. After the product is terminated, the registered email-id receives a PROVISIONED_PRODUCT_TERMINATED notification.

When you’re done, follow the procedures in Step  6 and Step 7 to clean up the resources created during this exercise.

Modify the default settings by setting ACTION to DISASSOCIATE

  1. Log in as administrator and repeat Step 1 to Step 4.
  2. On the Systems Manager Parameter Store dashboard:
    • Select the parameter that ends with /ACTION.
    • Choose Edit on top right side of the pane, change the value to DISASSOCIATE, and then choose Save Changes.
  3. Log out of the AWS Management Console as administrator.
  4. Follow Step 5.2 to 5.8 and launch the AWS Service Catalog product Amazon S3 Private Encrypted Bucket as an end user.
  5. Launch the same AWS Service Catalog product a second time. After the product is terminated, the registered email-id receives a DISASSOCIATED notification. The AWS Service Catalog product is disassociated after provisioning the product twice (MAXVALUE). The end user can no longer see or launch this product.
  6. Terminate one instance of the provisioned product. Notice that the AWS Service Catalog product is associated back to the portfolio, and the end user can use it again. The registered email-id receives an ASSOCIATED notification.

When you are done, follow the steps in Step 6 Delete a provisioned product and Step 7 Dissociate a product from a portfolio to clean up the resources created during this exercise.

Cleaning up

To avoid incurring costs, delete the resources created for this exercise. You already deleted the provisioned products and SCSetup. To finish the cleanup, you must also delete the AWS CloudFormation stack named AWSConfigSC.

Conclusion

In this post, you learned how to trigger notifications and take remediation actions automatically based on the events generated by AWS Service Catalog on any configuration changes. The ability to trigger definitive actions under distinct criteria helps organizations stay within their budget. It also helps them gain insight into the AWS resource utilization within their organization.

About the Author

Author Kishore

Kishore Vinjam is a partner solutions architect focusing on AWS Service Catalog, AWS Control Tower, and AWS Marketplace. He is passionate about working in cloud technologies, and working with customers and building solutions for them. When not working, he likes to spend time with his family, hike, and play volleyball and ping-pong.