How to Change SaaS Network Configurations Without Affecting Your Customers

By Ranjan Banerji, Sr. Partner Solutions Architect for Power and Utilities at AWS

Many organizations prefer to connect to software-as-a-service (SaaS) or managed services over a virtual private network (VPN). If you are an independent software vendor (ISV) that has to periodically change your network configuration, these customer VPNs present a challenge.

For example, if you change the classless inter-domain routing (CIDR) table of the virtual private cloud (VPC) that hosts your SaaS, your customers have to change their VPN settings. The same is true if you add a new VPC. This can take time, particularly if your customers belong to regulated or compliance-driven industries. The change management processes in these industries can take months to approve and implement changes.

Now, imagine hosting a multi-tenant service and having to wait for each customer to complete changes to their networks according to their change management processes. Not quite agile, is it? Naturally, you would prefer to change your networking configuration without waiting for your customers to change anything on their side.

I know of four ways to do just that:

• Deploy AWS Transit Gateway
• Switch AWS VPN from one VPC to another
• Add an edge VPC with proxy servers
• Add an edge VPC with load balancers

This post explains how to implement each one of these solutions. But first, let’s talk about which network changes at a SaaS or managed service affect the VPN connections of your customers.

Which SaaS Network Changes Affect Customer VPNs

If you need to change the network configuration of your SaaS or managed service, it’s usually for one of two reasons:

1. You are switching services from one VPC to another in a blue-green deployment. Usually, both the blue and green VPCs use the same CIDR block.
   .
2. You are expanding or adding services that require additional IP numbers, so you have to either modify the CIDR block of the existing VPC or add a new VPC.

In the blue-green deployment, both VPCs tend to have identical CIDR blocks, but only one at a time is accessible through a customer’s VPN connection. One VPC hosts what is currently in production, and the second is available to receive a new product release.

During a product release, you first deploy assets to the second VPC and then test them. When ready to go, this VPC becomes the target of the VPN connection.

Because both VPCs have identical CIDR blocks, this approach does not require any changes to the customer’s VPN configuration, or any changes to their networking and boundary security. However, if the CIDR blocks are not identical, this switchover can affect the VPNs at customer sites.

In the second scenario, where you are expanding or adding new services, the customer VPN configuration is affected. Because they may require different or additional IP numbers, you would have to either add a new CIDR block to the existing VPC or add a new VPC.

Solution 1: Deploy AWS Transit Gateway

AWS Transit Gateway is a good solution for a blue-green deployment when VPCs share identical CIDR blocks. It makes managing and switching routes an easy process. Traffic flow changes without having to make any configuration changes to the customer’s VPN setup.

Figure 1 – AWS Transit Gateway accommodates new CIDR blocks without affecting VPNs.

Let’s say these are the CIDRs for the VPCs hosting the customer environment and your products:

• Customer enterprise is 10.0.0.0/16
• ISV product 1 CIDR is 10.2.0.0/24
• ISV Product 2 CIDR is 10.3.0.0/24

In our blue-green deployment, you’d have four VPCs for products, plus one for other uses:

• Product 1 vpc-blue1 (10.2.0.0/24)
• Product 1 vpc-green1 (10.2.0.0/24)
• Product 2 vpc-blue2 (10.3.0.0/24)
• Product 2 vpc-green2 (10.3.0.0/24)
• Some other VPC vpc-other (10.4.0.0/24)

To deploy AWS Transit Gateway, follow these steps.

Step 1: Create an AWS Transit Gateway

Log on to the AWS Management Console, go to Services, and select VPC. Then select Transit Gateways.

Create a new transit gateway following these instructions in the AWS documentation.

Step 2: Attach VPCs to AWS Transit Gateway

Attach all four VPCs (vpc-blue1, vpc-blue2, vpc-green1, vpc-green2) to AWS Transit Gateway.

Step 3: Update AWS Transit Gateway route tables

Once the transit gateways are attached to the VPCs, go to Transit Gateway Route Tables and select the route table for your transit gateway.

Notice that a route has been added with a Route type of propagated. Ignore it.

Select Create route, and create a route to 10.2.0.0/24 (the CIDR for vpc-blue1 and vpc-green1). Then, select vpc-blue1 as the attachment. Notice the original route type propagated has been replaced with static:

Repeat this process for 10.3.0.0/24 (the CIDR for vpc-blue2 and vpc-green2), and any other VPC you are attaching to AWS Transit Gateway.

Step 4: Set up VPC route tables

For each VPC attached to the transit gateway, add a new route to send traffic destined for 10.0.0.0/16 to the AWS Transit Gateway. For example:

Step 5: Attach AWS Transit Gateway to VPN

Create a new attachment for AWS Transit Gateway. This time, select VPN instead of VPC. Make sure the AWS Transit Gateway route table has a route pointing traffic to 10.0.0.0/16 to the VPN. For example:

Step 6: Switch VPCs

From this point on, switching traffic over the VPN from vpc-blue to vpc-green is just a matter of updating the AWS Transit Gateway route table:

1. Select the route for 10.2.0.0/24 and select Replace route.
   .
2. Select the new target VPC (vpc-green1) and select Replace route.

Please note there is a cost associated with using AWS Transit Gateway.

Solution 2: Switch AWS VPN from One VPC to Another

Instead of using AWS Transit Gateway, you can simply detach the AWS VPN connection from one VPC in your blue-green deployment, and re-attach it to the other VPC.

Figure 2 – Switching AWS VPN from one VPC to another.

Let’s say this is your customer’s CIDR:

• 10.0.0.0/16 – customer enterprise CIDR

And these are the VPCs:

• 10.2.0.0/16 – vpc-blue
• 10.2.0.0/16 – vpc-green

The switchover would happen this way:

1. Customer establishes a VPN connection to vpc-blue.
   .
2. When you are ready to switch VPCs, you detach the AWS VPN from vpc-blue and attach it to vpc-green.

No changes are required on the customer’s enterprise side.

This is a more cost-effective approach than the first solution that uses AWS Transit Gateway. However, moving AWS VPN from one VPC to another disables the VPN connection for a few seconds. To decide between solutions, compare acceptable VPN downtime to the cost of AWS Transit Gateway.

There is another limitation to this approach. If you add new VPCs with different CIDRs, or if the CIDR of existing VPCs needs to be changed, the customer needs to make network configuration changes on their side.

Solution 3: Use an Edge VPC with Proxy Servers

This solution creates an edge VPC as a way to route a customer’s access to your SaaS service. The customer establishes a VPN connection to this edge VPC instead of the VPC hosting the SaaS application. A pair of highly available proxy servers route requests to the SaaS service.

With this solution, you can do more than a blue-green deployment. You can route customers to multiple services residing in different VPCs, thereby keeping the target VPC CIDR abstracted away from the customer.

Figure 3 – Using an edge VPC.

The proxy servers forward requests to services in the different VPCs. When you add new VPCs, modify a VPC’s CIDR, or do a blue-green switch, you simply reconfigure the proxy servers. The customer is completely abstracted away from the various target application VPCs.

Let’s say you are working with these CIDRs:

• 10.0.0.0/16 – Customer network CIDR
• 10.1.0.0/16 – Edge VPC CIDR

The goal is to avoid reconfiguring the VPN and its routes on the enterprise side of the customer once the initial connection is set up. Traffic, however, needs to be sent to services residing in multiple VPCs with varying CIDRs:

• 10.2.0.0/16 – vpc-blue
• 10.2.0.0/16 – vpc-green
• 10.5.0.0/16 – other VPC
• 172.1.0.0/16 – other VPC

To do that, follow these steps.

Step 1: Deploy proxy servers in the edge VPC

You can configure a highly available pair of proxy servers in two different AWS Availability Zones in the edge VPC.

Step 2: Create a peering connection from the edge VPC to each application VPC

Create a peering connection by following these instructions. Note that both your blue and green VPCs may be using the same CIDR.

Step 3: Update route tables in all VPCs

For example, the route table for the edge VPC has a unique route to each of the application VPCs, except where they share the same CIDR (blue-green). In this case, you should create single route:

Step 4: Switch VPCs

For a blue-green deployment, you can switch VPCs by simply updating the route table to the correct peering connection, provided the two VPCs have the same CIDR. If the two VPCs have different CIDRs, you can configure the proxy server to direct traffic to the appropriate application endpoint.

This approach gives you a lot of flexibility because it supports VPCs with CIDRs that may be modified, and it lets you add or remove VPCs. None of these changes result in any change to your customer’s networking configuration. However, the proxy servers run on Amazon Elastic Compute Cloud (Amazon EC2) instances, which incur a cost and require you to manage and maintain them.

Solution 4: Use an Edge VPC with Load Balancers

This solution uses an edge VPC like the previous solution does, but employs load balancers instead of proxy servers to direct traffic to the application VPCs. It uses AWS PrivateLink to provide connectivity between Application Load Balancers in the edge VPC and Network Load Balancers in the target VPCs.

Figure 4 – Using an edge VPC with load balancers.

To implement this solution, create a VPN connection to an edge VPC, which hosts an Application Load Balancer. The load balancer receives all of the traffic from the customer’s network. You can configure it to use AWS PrivateLink path-based routing to direct traffic to the Network Load Balancer.

The Network Load Balancer forwards requests to the appropriate target group of services in the application VPC. In this way, you can implement a blue-green deployment strategy or extend services to additional VPCs without impacting your customer’s network configuration.

Follow these instructions to securely publish internet applications at scale using Application Load Balancer and AWS PrivateLink.

The advantage of this approach is that you can switch VPCs (blue-green), add new services in new VPCs, and change VPC CIDRs without impacting your customer’s networking configuration. Another big advantage is that you don’t need to manage a pair of proxy servers. Application Load Balancer, Network Load Balancer, and AWS PrivateLink are all managed by AWS.

Comparing the Solutions

The first two solutions outlined in this post let you switch between blue-green environments whose VPCs use identical CIDR blocks. The first solution, which uses AWS Transit Gateway, does not interrupt service to the customer, but incurs costs.

The second solution, which switches AWS VPN from one VPC to another, does not incur costs (in all scenarios the cost of a VPN connection is assumed to be a given), but interrupts service to the customer, though for only seconds.

Neither of these solutions separates the application VPCs from your customers. If you have 50 customers, and the application VPC CIDR is 10.2.0.0/16, the likelihood that all 50 customers can support this CIDR is low. These approaches do not let you add additional VPCs with different CIDRs to offer new services without requiring a change in your customer’s network configuration.

The third and fourth solutions in this post use an edge VPC. One uses a pair of proxy servers, while the other uses AWS PrivateLink to route traffic to application load balancers and network load balancers. Both of these options are more flexible. They let you add new VPCs or modify the CIDRs of the application VPCs without impacting your customer networking configuration.

In addition to being, flexible, Solution 4 relies on AWS to maintain AWS PrivateLink and the load balancers. Unless there’s a specific use case that requires a proxy server, Solution 4 is more efficient from an operational point of view.

Conclusion

The preceding solutions describe different approaches to isolating virtual private clouds that host applications and services from an enterprise network. They all minimize or eliminate any changes on the enterprise side. You can use them to mitigate any cost or delay due to VPN connection reconfigurations.

You can also use these strategies to add more services to your customers with no changes to customer network configurations, avoiding delays.

Examine your own requirements and service level agreements (SLAs) to choose the solution that works best for you.