Observe dynamic sites with Amazon CloudWatch Synthetics and AWS Systems Manager Parameter Store

Overview

Maintaining and improving end user experience is key and as your business grows, the number of endpoints you need to observe can grow quickly. It can become more challenging and time consuming to build multiple canaries to observe them. This solution is designed to show how you can use a consistent and automated approach to observe multiple endpoints using a single canary.

You can use Amazon CloudWatch Synthetics to create canaries, configurable scripts that run on a schedule, and to monitor your endpoints and API operations. Canaries can mimic a user journey and perform the same actions as a customer, which makes it possible to continuously verify your customer experience even when you don’t have any customer traffic on your applications. By using canaries, you can discover issues before your customers do.

If your canary uses ‘syn-nodejs-puppeteer-3.1’ or later runtime version, you can use the Heartbeat Monitoring Blueprint and batch up to a maximum of five URLs in a single canary versus using multiple canaries to monitor a single URL. This blog will guide you on how to build canaries if you need to monitor more than 5 endpoints.

In this blog, we will walk you through the steps to create a canary that will import a parameter from the AWS Systems Manager (SSM) Parameter Store that contains a list of URLs that need to be monitored. The architectural diagram below describes how the canary works.

Architecture diagram

Fig 1. Architecture diagram describing the solution workflow

Solution overview

1. When we create a canary, it creates a Lambda function in the background.
2. We are using SSM Parameter Store to store a string with all the URLs we want to monitor.
3. We are creating an IAM role that has the appropriate permissions to access the parameter store.
4. When the canary executes, the lambda function fetches the URLs from the parameter store and then executes the test for all the URLs.

Walkthrough

1. Creating an SSM Parameter:

aws ssm put-parameter --name "url-list" --value "https://www.amazon.com,https://thinkwithwp.com/blogs/,https://docs.thinkwithwp.com/cloudfront/,https://docs.thinkwithwp.com/ec2/,https://docs.thinkwithwp.com/s3/,https://thinkwithwp.com/pricing/" --type String

2. Fetch the ARN of the parameter so that you can use it as the resource value in the command below:

aws ssm get-parameters —name "url-list"

3. Creating the canary:

• You can n­­ow create the Canary from the CloudWatch console.
• Use Inline Editor and paste the code below for the script.
• Runtime version is ‘syn-python-selenium-2.0’.
• Lambda handler value is “<name of the canary>.handler. Example, if the canary is called ‘url’ then the Lambda Handler value is ‘url.handler’
• Choose the option “Create a new role” while creating the canary.

from aws_synthetics.selenium import synthetics_webdriver as syn_webdriver
from aws_synthetics.common import synthetics_logger as logger
import boto3

def main():
    ssmParamName='url-list'
    client = boto3.client('ssm')   
    response = client.get_parameter(Name=ssmParamName)
    urlList=response['Parameter']['Value'].split(',')

    # Set screenshot option
    takeScreenshot = True
   
    browser = syn_webdriver.Chrome()
    for url in urlList:
        browser.get(url)
   
        if takeScreenshot:
            browser.save_screenshot("loaded.png")
   
        response_code = syn_webdriver.get_http_response(url)
        if not response_code or response_code < 200 or response_code > 299:
            raise Exception("Failed to load page!")
        logger.info("Canary successfully executed")

def handler(event, context):
    # user defined log statements using synthetics_logger
    logger.info("Selenium Python heartbeat canary")
    return main()

4. Create a json policy file (ssmPolicy.json), This policy will allow additional permissions required for canary to access SSM Parameter:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "ssm:GetParameter",
"Resource": "arn:aws:ssm:<region>:<account-id>:parameter/url-list"
}
]
}

Note: Please ensure to replace the Resource ARN field with the ARN we extracted in step [2] or replace the region you are operating in ex us-east-1, the AWS account number and the name of the SSM parameter you have created. In the above example the SSM parameter that was created in step [1] is called ‘url-list’.

5. Create an the IAM Policy from the policy document “ssmPolicy.json” we created in step [4] & attach this to the newly created Canary IAM Role:

aws iam put-role-policy --role-name <CanaryRoleName> --policy-name urlListSSMAccess --policy-document file://ssmPolicy.json

Note: Replace <CanaryRoleName> with the name of the role that your canary is using. You can find this under the configuration tab of the canary.

Cost

1. Canaries are charged per run. For example, you are charged $0.0012 per canary run in the us-east-region. More details on pricing can be found here.
2. Parameter Store: Standard parameters are available at no additional charge.

Cleaning up

After you have finished experimenting, and to avoid incurring extra charges, you might want to delete canaries you created as part of this article. To do so, follow the steps described in the delete canary documentation.

About the authors