AWS Cloud Operations Blog

Migrate your AWS OpsWorks Stacks to AWS Systems Manager

AWS OpsWorks Stacks is a managed configuration management service which helps customers deploy and manage Chef based applications on Amazon Elastic Compute Cloud (Amazon EC2). The service, which was launched in 2013, predates several popular AWS offerings, such as AWS Systems Manager. Some customers desire a deeper level of control and flexibility than possible using OpsWorks Stacks. For example, customers often want to use newer AWS offerings, such as Amazon EC2 Auto Scaling Groups, or take advantage of new operating systems, EC2 instance types, Amazon Elastic Block Store (Amazon EBS) volume types that are not supported by the service.

Migrating from OpsWorks Stacks to Systems Manager makes this possible by utilizing AWS CloudFormation to configure AWS supported services in a way that achieves similar capabilities provided by OpsWorks Stacks. Furthermore, customers can take advantage of CloudFormation to enable greater levels of customization.

Today, AWS is releasing migration tooling that helps customers migrate applications from OpsWorks Stacks to Systems Manager using CloudFormation. This post walks you through an example migration.

Migrating OpsWorks Stacks to Systems Manager

This post includes a migration script link which helps you migrate from OpsWorks Stacks to Systems Manager. The script takes in a source application stack or layer as input and generates an opinionated CloudFormation template. You can then provision or customize it to create a functional replacement. The script works by reading existing OpsWorks entity information and generating a CloudFormation template that achieves similar functions using native AWS capabilities such as Systems Manager, Auto Scaling Groups and more. The migration script will not affect the operation of your existing OpsWorks stack and layer since it creates a parallel application in Application Manager.

Using the migration involves the following steps:

  1. Download the script files and unzip the package.
  2. Migrate your OpsWorks Stack to Systems Manager.
  3. Validate the migration.

Prerequisites

For this walkthrough, you should have the following prerequisites setup in your local machine:

Download the script files and unzip the package

In this step, you will download the zip file that contains the migration script and all the relevant files.
Detailed steps for multiple operating systems can be found in the documentation.

  1. Download the zip file using the below command.
aws s3api get-object \
 --bucket export-opsworks-stacks-bucket-prod-us-east-1 \
 --key export_opsworks_stacks_script.zip export_opsworks_stacks_script.zip
  1. If your local machine is Linux, then install the unzip utility. Skip this step for Windows machine.
sudo apt-get install unzip
sudo yum install unzip
  1. Unzip the file using the appropriate command for your operating system.

For Linux:

unzip export_opsworks_stacks_script.zip

For Windows, run the below command in PowerShell:

Expand-Archive -LiteralPath <PathToZipFile> -DestinationPath <PathToDestination>

Migrate OpsWorks Layer to Systems Manager

This step runs the migration script and creates an application in Applications Manager.

  1. Start the pipenv shell using the below commands from Linux terminal or Windows PowerShell.
pipenv install -r requirements.txt
pipenv shell
  1. Migrate a layer from your OpsWorks stack using the below command. For parameters support, please refer to the README.md file which is part of the package downloaded above.
python3 stack_exporter.py \
  --layer-id <OPSWORKS_LAYER_ID> \
  --region <STACK_REGION>

Validate the migration

Take the following steps to validate the migration:

  1. The CloudFormation stack that the script deploys includes an Auto Scaling group. The auto scaling group will initially have zero running instances. Before increasing the number of running instances, you can review the resources created by the stack.
  2. Next, you can change the Auto Scaling group configuration and set the Desired, Minimum and Maximum values. This will spin up an EC2 instance using the Launch Template generated by the migration script.
  3. You can use this instance to validate whether the migration was successful. We recommend confirming health checks and connectivity via the Application Load Balancer as well as reviewing the Chef Logs in Amazon S3.
  4. You can monitor and run operations in your instances using a new Instances tab on the Application Manager page. For more information about working with the Instances tab, see Working with your application instances.
  5. To customize the migration, you can find information on the parameters accepted by the script here.

This release adds a new Instances tab in the Application Manager console that you can view to see a summary of your running instances.

Select the Instances tab in the Application Manager console for a summary of your running instances.

Use the new Instances tab to view the current state of your Instances.

Figure 1. New Instances tab to view the current state of your Instances.

Choose a specific instance to open the details panel for that instance. From the details panel you can select the Execution History of the associated AWS Systems Manager Run Command commands performed for that instance and debug a failed execution by choosing the Execution ID.

From the Execution History tab, you can choose an Execution ID to debug a failure.

Figure 2. Execution History tab to debug a failure.

Cleaning up

To avoid incurring future charges, delete the resources as described in README.md file, which is part of the package downloaded above.

AWS OpsWorks features mapped to AWS services post-migration

When you use Systems Manager to manage your applications, the underlying resources are no longer proprietary to OpsWorks. The table below shows how OpsWorks features have been mapped to AWS services.

Feature AWS OpsWorks AWS Systems Manager
Chef Runs Managed by AWS OpsWorks AWS Systems Manager Run Command: AWS-ApplyChefRecipes Command Document
Chef Parameters Managed by AWS OpsWorks SSM Parameter Store
Amazon EC2 Instance and EBS configurations Managed by AWS OpsWorks EC2 Launch Template
Auto Scaling Managed by AWS OpsWorks Auto Scaling Groups
EC2 Instance Life Cycle Managed by AWS OpsWorks Auto Scaling Groups Life Cycle Events
EC2 Instance Termination Protection Managed by AWS OpsWorks Auto Scaling Groups Custom Termination Protection
Load Balancing Classic Load Balancers Classic Load Balancers or Application Load Balancers
Infrastructure Definition and Provisioning Managed by AWS OpsWorks AWS CloudFormation
Chef Client Logs Managed by AWS OpsWorks Amazon S3, AWS CloudWatch Logs

Conclusion

In this blog, we have outlined a method for migrating your existing OpsWorks stack to Systems Manager by using the provided migration script. By migrating, you will provision your OpsWorks resources using AWS services such as Auto Scaling Groups, CloudFormation stacks, and then manage your resources using the Application Manager console and API’s. After migration, you can benefit from Application Manager’s capabilities such as tracking and remediating compliance issues, optimizing EC2 costs, viewing alarms and defining notification actions. For more information see the following resources:

About the authors:

Chetan Makvana

Chetan Makvana is a senior solutions architect working with global systems integrators at AWS. He works with AWS partners and customers to provide them with architectural guidance for building scalable architecture and execute strategies to drive adoption of AWS services. He is a technology enthusiast and a builder with a core area of interest on serverless and DevOps. Outside of work, he enjoys binge-watching, traveling and music.

Nishant Casey

Nishant Casey is a Senior Solutions Architect who primarily works with Not for Profit organizations in the UK and across EMEA. He works with partners and AWS internal teams in order to act as a technical advisor to his customers. He promotes architectural best practices and guides customers on how to design their systems to meet them. A DJ and gigging musician in his past life, Nishant is an avid home chef, baker and fan of RTS video games.

Avinash Karthik

Avinash Karthik is a Senior Software Engineer at AWS. He has been working with AWS technology for more than 5 years. Currently Avinash works on designing and implementing new features in OpsWorks and Systems Manager services. Outside of work he enjoys travelling, bouldering and tennis.

Chrisy Samara

Chrisy Samara is a Software Development Engineer at AWS with 12 years of experience in the software industry from Telecommunications, Computer-aided Engineering/Design to Cloud Services. Currently Chrisy works on designing and implementing new features in OpsWorks and Systems Manager services. She enjoys running, travelling and any discussion about databases and microservices.