Automating shared VPC deployments with AWS CloudFormation

VPC sharing allows customers to share subnets from a central AWS account with other AWS accounts in the same organization created in AWS Organizations. Centralized control of your virtual private cloud (VPC) structure allows you to maintain separation of duties through AWS account boundaries. A best practice for creating VPCs and other resources in the AWS Cloud is to develop your infrastructure as code (IaC). You can use AWS CloudFormation for this purpose because it offers an easy way to model, provision, and manage your resources through reusable templates.

In this post, I’ll show you how to create shared VPCs using AWS CloudFormation to support agility and the reusability of your workload resources. I’ll also share some considerations for when you combine the centralized network management capabilities of VPC sharing with teams building application workloads in AWS CloudFormation.

Solution implementation

To build the components of this solution, you will be working with AWS Organizations, AWS CloudFormation, AWS Resource Access Manager (AWS RAM), and AWS Systems Manager (SSM) Parameter Store. You will deploy and share network components from an AWS account you designate for network management using AWS CloudFormation and the AWS::CloudFormation::StackSet resource type.

Figure 1: Shared VPC Solution Diagram

Prerequisites

1. An organization created in AWS Organizations with a management account and at least two member accounts. One member account will be your network account and the other is designated for workload resources.
2. AWS Command Line Interface (AWS CLI) installed and configured for access to all organization accounts.
3. A text editor for updating YAML files.
4. An Amazon Simple Storage Service (Amazon S3) bucket accessible from your organization’s accounts. For an example of how to define an S3 bucket policy, see the An easier way to control access to AWS resources by using the AWS organization of IAM principals blog post.
5. The AWS CloudFormation template examples on GitHub.

Step 1: Set up IAM roles for cross-account access

To get started, create an AWS Identity and Access Management (IAM) role in the central network account and in each of member accounts that will be accessing the shared VPC components. You will use AWS CloudFormation StackSets to automate the deployment of the following roles from the management account of the organization.

• NetworkParameterExecutionRole: This role will be created in the workload accounts to allow for the execution of AWS CloudFormation stacks initiated from the network account.
• NetworkParameterAdminRole: This role will be created in the network account. It allows AWS CloudFormation to assume the NetworkParameterExecutionRole in the workload accounts.

The network-role-stackset.yaml template defines the resources for the two roles. It takes two parameter values: networkAccountId, which is used to set the AWS account that will be defined as the central network account, and organizationId. The template also uses the CloudFormation Conditions section to determine which role should be deployed based on whether the target account ID matches the networkAccountId value.

You can deploy the network-role-stackset.yaml template by setting your AWS CLI profile to your organization’s management account and running the following commands. Be sure to replace the region ID and account IDs with values from your environment.

aws cloudformation create-stack-set --stack-set-name network-parameter-role --parameters ParameterKey=networkAccountId,ParameterValue=network-account-id --template-body file://network-role-stackset.yaml
aws cloudformation create-stack-instances --stack-set-name network-parameter-role --accounts network-account-id workload-account1-id workload-account2-id --regions region-id

Step 2: Deploy the shared VPC

Again, you will use AWS CloudFormation to deploy your VPC and related resources from the network account. The shared-vpc.yaml template will:

1. Create a VPC with public, private, and database subnets across two Availability Zones.
2. Share the public, private, and database subnet tiers to the workload accounts you specify.
3. Create and share a prefix list to organization member accounts for use in security groups.
4. Create Parameter Store parameters with the shared configuration information in the workload accounts.

To deploy the shared VPC

1. Upload the ssm-parameter-stackset.yaml to your S3 bucket.
2. Edit the shared-vpc-input.json and provide the parameter values for your organization.
3. From the network account, use the following CLI command to create the stack.

aws cloudformation deploy --template-file shared-vpc.yaml --stack-name shared-vpc --parameter-overrides file://shared-vpc-input.json --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_NAMED_IAM CAPABILITY_IAM

Let’s take a closer look at how the template provisions and shares the VPC resources.

VPC sharing uses AWS RAM to share subnets with other member accounts in the organization. In the template, the AWS::RAM::ResourceShare resource type is used to create a RAM share for each of the three subnet tiers. AWS CloudFormation parameters are defined at the beginning of the template to capture comma-delimited lists of the AWS account IDs to share the subnet resources with. Using AWS CloudFormation parameters simplifies updates in the event you need to give access to other accounts in the future.

Parameters:
  vpcAccountIds:
    Type: String
    Description: Comma delimited list of Account ID requiring shared VPC access
  publicTierAccountIds:
    Type: String
    Description: Comma delimited list of Account ID requiring Public Subnet access
...
PublicSubnetsShare:
    Type: "AWS::RAM::ResourceShare"
    DependsOn:
        - PublicSubnet1Id
        - PublicSubnet2Id
    Properties:
      Name: "Public Subnet Shares"
      ResourceArns:
        - !Sub 'arn:aws:ec2:${AWS::Region}:${AWS::AccountId}:subnet/${PublicSubnet1}'
        - !Sub 'arn:aws:ec2:${AWS::Region}:${AWS::AccountId}:subnet/${PublicSubnet2}'
      Principals: !Split [ ",", !Ref publicTierAccountIds ]
      Tags:
        - Key: "Environment"
          Value: !Ref env

A prefix list is a set of one or more Classless Inter-Domain Routing (CIDR) blocks that can be referenced by other AWS resources, such as subnet route tables and VPC security groups. You can use AWS RAM to share prefix lists, which is useful in shared VPC environments. For example, a prefix list can be referenced in security groups for EC2 resources to allow access from trusted on-premises networks. Network teams can manage the prefix lists and application teams that use the shared subnets can reference the prefix lists rather than specifying the CIDRs directly in their templates. The Principals attribute of this RAM resource share differs from the VPC and subnet shares definitions. In this example, you are sharing the prefix list with the entire organization rather than just the workload accounts.

OnPremPrefixList:
    Type: AWS::EC2::PrefixList
    Properties:
      PrefixListName: "on-prem-networks"
      AddressFamily: "IPv4"
      MaxEntries: 10
      Entries:
        - Cidr: "10.100.1.0/24"
          Description: "Charlotte Office"
        - Cidr: "10.100.2.0/32"
          Description: "Seattle Office"
      Tags:
        - Key: "Name"
          Value: "Ops team networks"
        - Key: Environment
          Value: !Ref env
 ...
 OnPremPrefixListShare:
    Type: "AWS::RAM::ResourceShare"
    Properties:
      Name: "OnPrem Prefix List"
      ResourceArns:
        - !GetAtt OnPremPrefixList.Arn
      Principals: 
        - !Ref orgArn
      Tags:
        - Key: "Environment"
          Value: !Ref env

AWS CloudFormation stacks created in one account cannot export stack output values to another account. This means that if your teams are building resources in the workload accounts, they cannot use Fn::ImportValue to reference shared resource attributes such as a SubnetId. Manually coding or entering parameter values for these values is less efficient and prone to error. To automate the sharing of these values, you can define a naming standard to reference the resource IDs and publish them as a key-value pairs in the Parameter Store in the target account.

The AWS::CloudFormation::StackSet resource type is used to handle the cross account deployment of the parameters through the IAM roles you created earlier. Application teams can then reference the Parameter Store values using an environment naming convention when they define their templates. This method reduces human error and helps to promote reuse of the workload templates throughout the application lifecycle.

PublicSubnet1Id:
    Type: AWS::CloudFormation::StackSet
    DependsOn: PublicSubnet1
    Properties: 
      AdministrationRoleARN: !Sub arn:aws:iam::${AWS::AccountId}:role/NetworkParameterAdminRole
      Description: Publishes a parameter with PublicSubnet1 ID
      ExecutionRoleName: NetworkParameterExecutionRole
      OperationPreferences: 
          FailureToleranceCount: 1
          MaxConcurrentCount: 2
      Parameters: 
        - ParameterKey: Key
          ParameterValue: !Sub '/Network/${env}/PublicSubnet1'
        - ParameterKey: Description
          ParameterValue: !Sub 'Public Subnet 1 ID in ${env} VPC'
        - ParameterKey: Value
          ParameterValue: !Ref PublicSubnet1
        - ParameterKey: Type
          ParameterValue: String
        - ParameterKey: Env 
          ParameterValue: !Ref env
      PermissionModel: SELF_MANAGED
      StackInstancesGroup: 
        - DeploymentTargets:
            Accounts: !Split [ "," , !Ref publicTierAccountIds ]
          Regions:
            - !Ref 'AWS::Region'
      StackSetName: PublicSubnet1IdParameter
      Tags: 
        - Key: Environment
          Value: !Ref env
      TemplateURL: !Sub https://${s3Bucket}.s3.amazonaws.com/ssm-parameter-stackset.yaml

The StackSet resources use a common template, ssm-parameter-stackset.yaml, for defining the AWS::SSM::Parameter resources in the workload accounts you specify.

Step 3: Deploy workload resources

Now that you’ve successfully deployed the shared-vpc stack, you will find a collection of parameters in the Parameter Store of your workload accounts. The list in each account will be based on the components you assigned through the template parameters.

Figure 2: AWS Systems Manager Parameter Store

You can use these parameters as inputs to your workload templates to deploy resources into the shared subnets. In this case, you will use the workload-instance.yaml template to create an EC2 instance in one of the private subnets. This instance will also be configured with a security group that allows SSH access from the IP address ranges specified in the shared prefix list.

To deploy the workload instance

1. Edit the workload-instance-input.json and provide the parameter values for your application account.
2. Use the following CLI command to create the stack.

aws cloudformation deploy --template-file workload-instance.yaml --stack-name workload-instance --parameter-overrides file://workload-instance-input.json --capabilities CAPABILITY_IAM

After your stack is created, use the CloudFormation console or the following CLI command to confirm your instance details.

aws ec2 describe-instances --filter "Name=tag:Name,Values=AppServer1-Dev"

Let’s take a closer look at how the template uses the shared resources to provision EC2 resources.

The shared-vpc.yaml template you deployed earlier created Parameter Store parameters for the VPC ID, subnet IDs, and the prefix list ID. The workload-instance.yaml template retrieves this information from the Parameter Store in the Parameter section of the template. The AWS::SSM::Parameter::Value<String> parameter type enables this functionality.

  onPremPrefixList:
    Type: AWS::SSM::Parameter::Value<String>
    Description: SSM Parameter Store key for Prefix-List
    AllowedValues:
      - /Network/Dev/OnPrem-PrefixList
      - /Network/Test/OnPrem-PrefixList
      - /Network/Prod/OnPrem-PrefixList
  vpc:
    Type: AWS::SSM::Parameter::Value<String>
    Description: SSM Parameter Store key for VPC ID
    AllowedValues:
      - /Network/Dev/Vpc
      - /Network/Test/Vpc
      - /Network/Prod/Vpc
  subnet:
    Type: AWS::SSM::Parameter::Value<String>
    Description: SSM Parameter Store key for VPC ID
    AllowedValues:
      - /Network/Dev/PrivateSubnet1
      - /Network/Test/PrivateSubnet1
      - /Network/Prod/PrivateSubnet1

These parameter values are then referenced by the resources in the template as needed.

Cleanup

Set your CLI profile to the application account and use the following commands to delete the resources deployed to this account.

aws cloudformation delete-stack --stack-name workload-instance

Then, switch your CLI profile to the network account and use the following commands to delete the resources deployed to this account.

aws cloudformation delete-stack --stack-name shared-vpc

Then, switch your CLI profile to the organization’s management account and use the following commands to delete the resources deployed to this account. Update the account ID and region ID values to match those you used during your deployment.

aws cloudformation delete-stack-instances --stack-set-name network-parameter-role --accounts network-account-id app-account1-id app-account2-id --no-retain-stacks --regions region-id  
aws cloudformation delete-stack-set --stack-set-name network-parameter-role

Conclusion

In this post, I shared an example of how to use AWS CloudFormation to provision and share VPC resources between teams in a multi-account model. I showed you how to create the IAM roles and deployment framework required to support a centralized networking model that follows the best practices. Through upfront planning and the use of AWS CloudFormation StackSets and AWS Systems Manager Parameter Store, workload teams can remain agile in a centralized environment.

About the author