Schedule modifications of Amazon RDS using Amazon EventBridge Scheduler and AWS Lambda

Amazon Relational Database Service (Amazon RDS) makes it straightforward to set up, operate, and scale a relational database in the cloud. Traditional relational databases require time spent on capacity planning, maintenance, backup, and recovery; a substantial amount of a database administrator’s time is lost to these tasks. Amazon RDS helps DBAs focus on other important tasks that add value to the organization by automating most routine tasks.

In this post, we present a solution using Amazon EventBridge Scheduler and AWS Lambda that allows you to schedule a programmatic modification of a DB instance with specific tags.

Solution overview

Amazon RDS provides different instance types optimized to fit different relational database use cases. You can modify provisioned instances manually from the Amazon RDS console or using an API. When modifications need to be done on a recurring basis, such as scaling an instance up and down during predefined periods of time, you can automate the task using EventBridge Scheduler and Lambda.

Lambda is a compute service that lets you run code without managing any servers. You don’t have to worry about provisioning servers, configuring the operating systems, installing applications, and so on.

EventBridge Scheduler is a serverless scheduler that allows you to create, run, and manage tasks from one central, managed service. With EventBridge Scheduler, you can create schedules using cron and rate expressions for recurring patterns, or configure one-time invocations. You can set up flexible time windows for delivery, define retry limits, and set the maximum retention time for failed API invocations.

In this proposed solution, we use a Lambda function to store the code that modifies an RDS instance that has the tag Scaling, and use EventBridge Scheduler to invoke the Lambda function.

The following diagram illustrates solution architecture.

To implement the solution, you need to complete the following high-level steps:

1. Create tags for your RDS DB instances. The tags are used to identify the instances in the scope of change.
2. Create an AWS Identity and Access Management (IAM) policy and role for Lambda.
3. Create a Lambda function with relevant parameters to modify your database.
4. Create a schedule to invoke the Lambda function as needed.

Prerequisites

To follow the steps in this post, you need the following:

• An AWS account with administrator access to Amazon RDS and permissions to manage IAM policies and roles, Lambda functions and EventBridge schedules.
• An RDS instance that you want to modify on a schedule.

Create tags for your RDS DB instances

You can assign tags while creating a DB instance or by modifying the instance after it’s created. The following steps walk you through assigning tags for a scheduled stop and start:

1. On the Amazon RDS console, choose a DB instance that you want to add tags to.
2. On the Tags tab, choose Manage tags.
3. Choose Add new tag.
   
4. For Tag key, enter Scaling.
5. For Value, enter up-down.
6. Choose Save changes.

You can now see the added tags on the Tags tab.

Create an IAM policy and role for Lambda

Complete the following steps to create an IAM policy and role for Lambda to modify DB instances:

1. On the IAM console, under Access management in the navigation pane, choose Policies.
2. Choose Create policy.
3. On the JSON tab, enter the following policy code:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": [
                   "rds:DescribeDBClusterParameters",
                   "rds:DescribeDBEngineVersions",
                   "rds:DescribeGlobalClusters",
                   "rds:DescribePendingMaintenanceActions",
                   "rds:DescribeDBLogFiles",
                   "rds:DescribeReservedDBInstancesOfferings",
                   "rds:DescribeReservedDBInstances",
                   "rds:ListTagsForResource",
                   "rds:DescribeValidDBInstanceModifications",
                   "rds:DescribeDBInstances",
                   "rds:DescribeSourceRegions",
                   "rds:DescribeDBClusterEndpoints",
                   "rds:DescribeDBClusters",
                   "rds:DescribeDBClusterParameterGroups",
                   "rds:DescribeOptionGroups",
                   "rds:ModifyDBInstance"
               ],
               "Resource": "*"
           }
       ]
   }

   This policy allows modification of any RDS instance. You might want to apply least privilege principle and limit its scope by specifying your RDS DB instances’ Amazon Resource Names (ARNs) in the Resource instead of “*”.

4. Choose Next.
5. For Policy name enter rds-modify-instance.
6. Choose Create policy.
   
   Next, you create the IAM role.
7. On the IAM console, under Access management in the navigation pane, choose Roles.
8. Choose Create role.
9. For Select trusted entity, select AWS service.
10. For Use case, choose Lambda.
11. Choose Next.
12. Search for and select the policy you created (rds-modify-instance).
13. Search for and select the AWSLambdaBasicExecutionRole managed policy.
14. Choose Next.
15. For Role name, enter rds-lambda.
16. Review the attached policies and choose Create role.

You can now attach this role while creating your Lambda function.

Create your Lambda function to modify the database

For this post, we create a Lambda function that will be called to modify the DB instance type. For the full list of modifiable settings, refer to the API documentation.

1. On the Lambda console, in the navigation pane, choose Functions.
2. Choose Create function.
3. For Function name, enter rds-modify.
4. For Runtime, choose Python 3.12.
5. Expand the Change default execution role section and select Use an existing role.
6. For Existing role, choose the role you created (rds-lambda).
7. Choose Create function.
8. On the function details page, navigate to the function code section.
9. Delete the sample code and enter the following:

   import boto3
   import os
   import sys
   import time
   from datetime import datetime, timezone
   from time import gmtime, strftime
   
   def modify_rds(instance_name, instance_type, region):
       key=os.environ['KEY']
       value=os.environ['VALUE']
       in_the_group = False
       
       client = boto3.client('rds', region_name = region)
       try:
           response = client.describe_db_instances(DBInstanceIdentifier = instance_name)
   
           instance = response['DBInstances'][0]
           arn = instance['DBInstanceArn']
           resp2=client.list_tags_for_resource(ResourceName=arn)
           # Check if the RDS instance is part of the scaling group
           for tag in resp2['TagList']:
           # If the tags match, then modify instances by validating the current status.
               if tag['Key']==key and tag['Value'] == value:
                   in_the_group = True
                   break
           if in_the_group:
               if instance['DBInstanceStatus'] == 'available':
                   # Check if the requested instace type different from the existing one
                   if instance['DBInstanceClass'] == instance_type:
                       print('the DB instance is already running on the requested instance type{}'.format(instance_type))
                   else:
                       client.modify_db_instance(DBInstanceIdentifier = instance['DBInstanceIdentifier'], DBInstanceClass = instance_type, ApplyImmediately=True)
                       print('scaling DB instance {0}'.format(instance['DBInstanceIdentifier']), 'from {0}'.format(instance['DBInstanceClass']), 'to {}'.format(instance_type))
               elif instance['DBInstanceStatus'] == 'modifying':
                   print('DB Instance {0} is in modifying state and cannot be changed'.format(instance['DBInstanceIdentifier']))
               elif instance['DBInstanceStatus'] == 'stopped':
                   print('DB Instance {0} is stopped and cannot be modified'.format(instance['DBInstanceIdentifier']))
               elif instance['DBInstanceStatus'] == 'starting':
                   print('DB Instance {0} is in starting state. Wait until it is available and run the scaling again.'.format(instance['DBInstanceIdentifier']))
               elif instance['DBInstanceStatus'] == 'stopping':
                   print('DB Instance {0} is in stopping state.'.format(instance['DBInstanceIdentifier']))
           else:
                print('DB Instance {0} is not part of scaling target'.format(instance['DBInstanceIdentifier']))
   
       except client.exceptions.DBInstanceNotFoundFault:
           print('no such DB instance {}'.format(instance_name), 'in the region {}'.format(region))
   
   
   def lambda_handler(event, context):
       modify_rds(event['instance_to_update'], event['new_instance_type'], event['instance_region'])

10. Choose Deploy.The preceding Lambda function needs two parameters (KEY, VALUE) to be passed as environment variables. KEY and VALUE are the tags that we have attached to the instances that require scaling in the previous steps. The values that we attached to the RDS instances should match exactly to the environment variables.
11. On the Configuration tab, choose General configuration section.
12. Choose Edit and change Timeout to 5 seconds.
13. On the Configuration tab, in the Environment variables section, choose Edit and add the environment variables as shown in the following screenshot.
14. Choose Save.
15. Choose the Test tab to test the function.
16. For Event JSON, you need to provide three input parameters expected by the function and their values:
    a. instance_to_update – The DB identifier of the instance to be modified.
    b. new_instance_type – The instance type to modify the DB instance for.
    c. instance_region – The AWS Region in which the DB instance resides.

    {
      "instance_to_update": "database-1",
      "new_instance_type": "db.m5.xlarge",
      "instance_region": "us-east-1"
    }

    

17. Choose Test to test the function.The test runs the Lambda function that validates whether the database exists and is modifiable, then applies the change if applicable. You can see on the function detail page that the function was successful but the database was not modified due to being stopped.

After you create the function and test it, you can create a schedule to invoke the function as needed.

Create a schedule using EventBridge Scheduler

For this post, we configure a scheduler to run at the defined time to invoke the function.

1. On the EventBridge console, in the navigation pane, under Scheduler, choose Schedules.
2. Choose Create schedule.
3. For Schedule name, enter rds-modify-shedule.
4. For Occurrence, select your desired schedule. For this post, select One-time schedule.
5. For Date and time, enter today’s date and the time as 10 minutes in the future.
6. For Flexible time window, choose Off.
7. Choose Next.
8. For Target API, select AWS Lambda.
9. For Lambda function, choose the function created in the previous step (rds-modify).
10. For Payload, provide the input parameters for the function, the same way you did for the test event:

    {
      "instance_to_update": "database-1",
      "new_instance_type": "db.m5.xlarge",
      "instance_region": "us-east-1"
    }

11. Choose Next.
12. For Action after schedule completion, choose NONE.
13. For Retry policy, disable Retry.
14. For Execution role, choose Create new role for this schedule.
15. Choose Next.
16. Review the schedule and choose Create schedule.

The schedule will run on the defined time and invoke the function to modify the database. You can verify the new instance type on the Amazon RDS console and check the function status using Amazon CloudWatch logs.

Cleaning up

You can now delete the resources that you created, unless you want to retain them. By deleting AWS resources that you are no longer using, you prevent unnecessary charges to your AWS account.

1. To delete EventBridge rule and Lambda function, follow the steps in the Clean up your resources sections of the Amazon EventBridge user guide.
2. To delete tags from a DB instance, follow the steps in the Adding and deleting tags in Amazon RDS section of the Amazon RDS user guide.
3. To delete role for Lambda and IAM policy, follow Delete roles or instance profiles and Delete IAM policies sections in the AWS Identity and Access Management user guide.

Summary

In this post, we demonstrated how to schedule a Lambda function to programmatically modify an RDS DB instance at a predefined time. The benefits of automating modifications allow organizations to further reduce manual intervention and human effort while improving operational excellence. This includes auto upsizing DB instance prior to peak usage times and cost savings by downsizing the DB instance during low usage times.

We encourage you to try this solution and take advantage of all the benefits of using Lambda and Amazon RDS.

Reach out to share your feedback and questions in the comments section.

About the Author

Slava Beylin is Senior Technical Account Manager at AWS who works with customers to optimize their use of AWS services. Slava has 20 years of experience leading DevOps with a focus on relational databases and enterprise application.