Automate folder recovery with AWS Systems Manager

Amazon Elastic Block Store (Amazon EBS) snapshots provide you with a mechanism to back-up your critical workloads. There are several ways that you can back up your workloads, including Amazon Data Lifecycle Manager and AWS Backup. You can use Amazon Data Lifecycle Manager to automate the creation, retention, and deletion of Amazon EBS snapshots and Amazon EBS-backed AMIs.

These backups create Amazon Elastic Compute Cloud (Amazon EC2) Amazon Machine Image (AMI) and snapshots of your instance(s) volumes. Furthermore, if you’re in a recovery situation where you must restore the entire machine, then you can easily do so with a few clicks.

But what if you only need to restore a few files, not the entire instance?

Recovering a folder from an instance can be a long and tedious process because of the number of steps required. Normally, you would need to create a volume from a snapshot, mount the volume to an existing instance (or create a new one), find the folder, compress the folder, upload the folder to a shared location (likely Amazon Simple Storage Service (Amazon S3)). Then, you unmount and delete the volume from your instance. This is a time-consuming task, and one that many administrators find challenging.

In this post, we’ll demonstrate on how to deploy the AWS Systems Manager Automation runbook that will automate the process of retrieving a folder from a given snapshot, compress, and upload the zip file to a specified S3 bucket for both Windows and Linux. In addition, this automation runbook will also cleanup all of the deployed or created resources except for the recovered S3 object. This Automation runbook assumes that you know the folder, path, snapshot, and for Linux, the partition where the folder is located.

Prerequisites

The following prerequisites are necessary for this post:

Use of AWS CloudFormation and permissions to deploy a CloudFormation stack with the ability to create an AWS Identity and Access Management (IAM) role and instance profile.
Amazon Virtual Private Cloud (Amazon VPC) and Subnet with internet access.
Use of S3 Bucket as the destination for recovered zipped folders. We recommend using the least-privilege model for the S3 bucket by applying S3 server-side encryption enabled (SSE-S3) and S3 bucket policy.

Solution Walkthrough

Deployment of this solution will be done using CloudFormation, where two Systems Manager automation runbooks gets deployed, one for Windows and the other for Linux. Using these runbooks, we’ll recover a folder that you specify from a given snapshot and will compress the folder as a zip before place it in your S3 bucket.

Download CloudFormation template

git clone https://github.com/aws-samples/aws-systems-manager-folder-recovery.git

cd aws-systems-manager-folder-recovery

Deploy CloudFormation template

The following command deploys the resources required to run this solution.

aws cloudformation deploy --template-file AWSSystemsManagerFolderRecovery.yaml\
--stack-name automated-item-recovery \
--capabilities CAPABILITY_NAMED_IAM \
--region <deployment region> \
--parameter-overrides \
VPC=<desired VPC ID> \
Subnet=<desired subnet ID in above VPC> \
S3RecoveredBucket=<S3 bucket name where recovered items will be stored>

Recover Windows directory

In the following example, we have installed IIS on a Windows 2019 server. In our example, we will recover the C:\INETPUB\wwwroot directory so that we can take a look at our website content from a snapshot that was previously created.

$Screenshot showing a “tree” view of the directory structure for C:\INETPUB with an arrow indicating the C:\INETPUB\wwwroot directory as the one that we want to recover.$

Run the following command to recover the directory path from the prior example. Note that you must specify your directory and your snapshot to use to replicate this.

You’ll notice in the following command example that I want to recover the folder C:\INETPUB\wwwroot. However, we only put INETPUB\wwwroot, intentionally dropping the drive letter from the path.

aws ssm start-automation-execution \
--document-name Automated-Windows-Folder-Recovery \
--parameters SnapshotId="snap-0f7151a15ccc17xxx",RecoverFolder="inetpub\wwwroot"

From the console, we can observe the successful execution, and we can also see the individual steps the automation executed on our behalf. Within the AWS Console navigate to, AWS Systems Manager → Automation → Execution ID (find the ID returned by the above command).

Screenshot showing the steps in the automation document and their execution status on the AWS console, in this case the status is success.

If you select Step 3 Step ID, you can see the output for the Windows task. We have highlighted the text in the output where we see the directory being zipped, and we can also see where the zip is placed on Amazon S3.

Screenshot of command output showing AWS console with output from the command execution, creation of volume, attachment, mounting of volume, zipping of folder, detachment of volume, deletion of volume, and upload of zip containing folder.

Let’s look at the contents of the zip file which was uploaded by the automation. We can see zip file and the expanded contents containing the same files from the original screenshot. This proves us that the automation worked.

Screenshot showing the expanded zip file with the same two files iisstart.png and iisstart.htm that we wanted to recover from the beginning of the example.

Recover Linux directory

We have installed WordPress on a Amazon Linux 2 machine, but I’ve reduced the number of files and folders to make the output more readable. We’ll recover the /var/www/html folder using a previously created Amazon EC2 snapshot. Here’s what the folder structure looks like:

Screenshot showing a “tree” view of the directory structure for /var/www with a depth of two.

Run the below command for recovery. Note that you must specify your directory, snapshot, and partition to replicate.

In the following example, the folder path that we want to recover is /var/www/html. However, We have only put var/www/html, intentionally dropping the preceding “/” for this automation.

aws ssm start-automation-execution \
--document-name Automated-Linux-Folder-Recovery \
--parameters SnapshotId="snap-0f48aea559e50xxx",RecoverFolder="var/www/html",PartitionNumber="1"

From the AWS console, we can see the execution was successful, and we can see the individual steps the automation executed on our behalf. Within the AWS Console navigate to, AWS Systems Manager → Automation → Execution ID (find the ID returned by the prior command).

Screenshot showing the steps in the automation document and their execution status on the AWS console, in this case the status is success.

If we select Step 3 Step ID, we can see the output for our Linux task. We have highlighted the text in the output where we see the directory being zipped, and we can also see where the zip is placed on Amazon S3.

Screenshot of the command output showing the AWS console output from the command execution, creation of volume, attachment, mounting of volume, zipping of folder, detachment of volume, deletion of volume, and upload of zip containing folder.

Screenshot showing the expanded zip file with the same folder and files that we wanted to recover from the beginning of the example.

Cleanup

To cleanup this deployment, we must delete the CloudFormation template which we deployed earlier using the below command:

aws cloudformation delete-stack \
--stack-name automated-item-recovery \
--region <deployment region>

Delete the S3 bucket, Amazon VPC and Subnets that you created as part of the prerequisites for this post.

Conclusion

In the above example, we demonstrated on how you can deploy custom Systems Manager automation documents to automate the process of recovering individual folders from both Windows and Linux. Using this automated approach, you can reduce the steps required to recover files, while also keeping the complexity to a minimum, thus enabling any user to perform these actions. In this post, we deployed the solution on a single AWS account, but adapting this approach on a multi-account deployment can be achieved by using CloudFormation StackSets or Customizations for AWS Control Tower.

AWS Cloud Operations Blog