Managing backups at scale in your AWS Organizations using AWS Backup

Customers want the ability to have a standardized way to manage their backups at scale with AWS Backup and their AWS Organizations. AWS Backup offers a centralized, managed service to back up data across AWS services in the cloud and on premises using AWS Storage Gateway. AWS Backup serves as a single dashboard for backup, restore, and policy-based retention of different AWS resources, which include:

• Amazon EBS volumes
• Amazon EC2 instances
• Amazon RDS databases
• Amazon Aurora clusters
• Amazon DynamoDB tables
• Amazon EFS file systems
• AWS Storage Gateway volumes

With customers scaling up their AWS workloads across hundreds, if not thousands of AWS accounts, customers have expressed the need to centrally manage and monitor their backups. AWS Backup has partnered with AWS Organizations to launch a new cross-account management functionality that enables customers to centrally manage and monitor their backups across their AWS accounts directly from the management account of their organizations. Administrators that previously had to manually duplicate backup configurations across thousands of accounts, can now manage and monitor backups through a single process, from a single management account.

Customers can deploy an organization-wide backup plan to ensure compliance across all of their selected accounts in their AWS Organizations. This enables customers to standardize the way they implement backup policies, minimizing the risk for errors as well reducing the manual overhead. The cross-account management functionality from AWS Backup via AWS Organizations also facilitates greater transparency when it comes to meeting data protection regulations because a single backup policy can now be systematically applied across accounts. Customers can easily identify resources or accounts that have fallen out of compliance in minutes with a centralized view straight from the AWS Backup console.

In this blog, we discuss customer use cases that the new cross-account management functionality from AWS Backup via AWS Organizations solves. Additionally, we provide a step-by-step guide on getting started.

Customers want multi-account backup management

To begin, we discuss some different types of customers and why cross-account management is important to them.

Financial services customer

One of our largest financial services customers commented that the overhead and complexity of managing backup increases geometrically when you have multiple backup plans per account or organizational unit (OU, a collection of accounts in an organization). The customer approached this problem by deploying AWS CloudFormation templates for uniformity and scalability across accounts. While AWS CloudFormation provided a way to standardize backup plans across accounts, the customer also needed a scalable way to manage change sets among hundreds or even thousands of accounts.

Media and entertainment customer

We’ve also found our customers developing workarounds to ensure all of their accounts remain in compliance with corporate policies. One leading media and entertainment customer had to create and deploy their own rollout mechanisms for changes to protect their accounts through scripts and databases. The customer provided feedback that they would prefer to use a native AWS solution to track which accounts have been included or excluded, which accounts received the deployment, and which accounts are out of compliance. Enacting a change in a policy would often result in complex troubleshooting when the change failed to deploy to certain accounts.

ISV customer

Along with the deployment challenges, monitoring of backup compliance is also a challenge for some of our customers, especially as the number of accounts under management keeps increasing. With a handful of accounts, simply logging into the Console and checking status may be sufficient. But in an AWS environment with hundreds, or even thousands of accounts, this becomes quickly untenable. Some customers have deployed third-party solutions with a single-view dashboard. Others have built a notification structure to monitor backup status from a central account via Amazon SNS and Amazon SQS.

Customers benefit from using cross-account management

Cross-account management from AWS Backup via AWS Organizations allows customers to centrally manage, maintain, and monitor backups across their accounts. Let’s look at some typical use-cases for backup and restore in multi-account environments.

Centralized management

As enterprises adopt AWS at an accelerated pace, most customers use a multi-account strategy to isolate their resources and improve security. AWS Organizations allows customers to set up, group and centrally manage large numbers of AWS accounts. This enables them to take advantage of the natural access privilege boundaries inherent to an AWS account, while maintaining centralized control from a single account – the management account. With the new cross-account management functionality, customers can leverage the compliance and data protection functionality of AWS Backup, with the central management and governance capabilities from AWS Organizations. They are now able to centrally manage and monitor their backups across all of their accounts from the management account.

IT Governance

As part of IT governance, many customers first categorize all of their data into tiers based on how critical the data is. Each data tier has unique data protection requirements, including the Recovery Point Objective (RPO) and the backup retention period. Here’s one such example:

AWS Backup enables customers to package business requirements including RPO and retention period into a backup plan within a single AWS account. However, customers with a large number of accounts also need the ability to propagate a standardized backup plan for each data tier across all of their accounts reliably and securely. As new applications are onboarded and new AWS accounts are added, IT governance policies require consistency when applying backup plans. Cross-account management from AWS Backup via AWS Organizations enables customers to automatically apply backup policies to new applications and accounts.

Regulatory compliance

Many customers must meet compliance standards and are subject to oversight from regulatory bodies specific to their industry. Standards for backup and restore capabilities are often dictated by regulations. The backup implementation must be standardized and automated, especially in a complex environment where data or resources are spread across multiple accounts. Automation minimizes the risk of error from manual processes, which can result in compliance violations.

In addition, regulations require the ability to produce an audit trail of backup jobs in every account and completion status. With cross-account management from AWS Backup via AWS Organizations, customers can log into the management account and have a single dashboard view of backup operations as they occur across accounts. Cross-account management also enables customers to apply an immutable backup policy across accounts. Having an immutable shared policy creates a guarantee that the selected accounts are on the same data protection schedule. Customers can also quickly identify anomalies and rectify them as needed. They can also generate a single trail for all of their backup and restore jobs across their entire organization, making it simple to comply with regulatory needs.

Enhanced security

As customers move more of their mission-critical workloads onto AWS, they also expect that their data will be protected and secured against potential ransomware attacks and rogue actors. By enabling customers to manage their backups in an automated manner across their accounts via AWS Backup, AWS Identity and Access Management (IAM), and AWS Organizations provide features to mitigate these risks.

Let’s take a closer look at how you can apply fine-grained access control and secure your data using these services together. In AWS Backup, a backup vault is a container for backups. These could be backups from any supported AWS services, such as Amazon EBS snapshots or DynamoDB backups, and they are collectively referred to as “recovery points.” IAM policies can be used to regulate which IAM users and roles can create recovery points in a vault, restore from recovery points, or even delete them. As an example, even an EC2 administrator who has been granted ec2:* permissions will not be able to act on an AWS Backup recovery point unless given that permission within AWS Backup. To learn more about the security controls for AWS Backup, please see our documentation.

Customers can further protect all created recovery points by globally denying privileges to certain API actions such as “DeleteRecoveryPoint” via Service Control Policies (SCPs) that they can apply to any accounts within the organizational unit (OU) of their choice.

Setting up AWS Backup cross-account management and monitoring:

Before beginning, make sure that your AWS organization has all features enabled.

Enable cross-account management for AWS Backup via AWS Organizations

To get started, sign into your management account and go to the AWS Backup console. On the Settings page, enable Backup policies.

You can now create your first backup policy on the backup policy page under My organization.

Create a new backup policy

When creating a backup policy, a key component is adding a backup plan. Keep in mind that a backup policy cannot span multiple AWS Regions. Create a backup plan with the resource selection, via either the Visual Editor or by inserting a JSON template.

Now that your backup policy is created you are ready to attach it to accounts or even organizational units in your organization by clicking into your newly created backup policy.

Attach a target to the backup policy

From the backup policy detail page, you can attach your policy to individual accounts, OUs, or your entire organization. Select Attach in the Targets section at the bottom of the page.

Clicking Attach opens a model that displays a tree view of all individual accounts and OUs in your organization. Selecting Root attaches your policy to all accounts in your org, and selecting an OU attaches your policy to all sub-OUs and accounts within it. If you must get granular, your policy can be attached to a single account.

The backup policy has been attached to the selected target

Once your policy is attached to a target, the backup plan you created in your backup policy is automatically added to accounts in your selection. Any changes you make to your backup policy is automatically applied to the backup plan in the attached accounts. In the event an account joins a selected OU, it receives the backup policy automatically, and likewise, if an account leaves the selected OU, the previously effective backup policy no longer applies.

Additional configuration:

To further enhance the security across multiple accounts and Regions with a single operation, configure backup vault, AWS KMS, and IAM using AWS CloudFormation StackSets.

The following is a sample CloudFormation Stack.

Resources:
  KMSKey:
    Type: AWS::KMS::Key
    Properties:
      Description: "Encryption key for daily"
      EnableKeyRotation: True
      Enabled: True
      KeyPolicy:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            "AWS": { "Fn::Sub": "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" }
          Action:
          - kms:*
          Resource: "*"

  BackupVaultWithDailyBackups:
    Type: "AWS::Backup::BackupVault"
    Properties:
      BackupVaultName: "DistributedVault"
      EncryptionKeyArn: !GetAtt KMSKey.Arn

  BackupRole:
    Type: "AWS::IAM::Role"
    Properties:
     AssumeRolePolicyDocument:
       Version: "2012-10-17"
       Statement:
         -
          Effect: "Allow"
          Principal:
            Service:
              - "backup.amazonaws.com"
          Action:
            - "sts:AssumeRole"
     ManagedPolicyArns:
       -
        "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup"

Monitoring in the management account

On the cross-account monitoring page, advanced filters can be applied to monitor the status of jobs created across your organization down to a specific account and even Backup job ID.

Conclusion

In this blog post, we described the customer motivation behind cross-account management from AWS Backup via AWS Organizations. We also provided a detailed step-by-step guide of how the new feature works to centrally manage and monitor your backups in a multi-account environment. Cross-account management from AWS Backup via AWS Organizations greatly reduces the operational burden of protecting data and ensures that resources across all accounts are protected consistently. Most importantly, cross-account management minimizes the risk of compliance issues caused by mis-configured backup policies.

Thank you for reading this blog post discussing cross-account management from AWS Backup via AWS Organizations. Please share any comments you may have in the comments section.