AWS Cloud Operations Blog

Replacing SSH access to reduce management and security overhead with AWS Systems Manager

Cesar Soares, DevOps and cloud infrastructure manager, VR Beneficios

In many corporate enterprises, interactive shell access to cloud or datacenter environments is a necessity. It must be supported in a secure, auditable manner, often programmatic or via scripting, and with strong access controls. As discussed in a previous post by Jeff Barr, AWS Systems Manager Session Manager is just the tool to meet these business requirements.

This post describes how VR Beneficios, a large, Brazil-based benefits company with over 40 years of industry experience, replaced all its SSH access with the secure interactive shell access provided by Session Manager.

Overview
Interactive shell access to traditional server-based resources often comes with high management and security overhead. The reason is that user accounts, passwords, SSH keys, and inbound network ports need to be maintained to provide this level of access. Often, there is also the cost of supporting additional infrastructure for bastion hosts, which is a common way of creating a security boundary between less-secure to more-secure resources.

The conversation tends to become more complex when additional security or functional requirements are needed, because of this type functionality is usually not natively supported. These requirements include auditability, access control, single sign-on—or, as is often seen with AWS customers—programmatic access to the resources to leverage scripting or automation.

Solution
After evaluating various options, VR Beneficios decided to use Session Manager because it solved the business problems outlined earlier, including the seamless and programmatic access. The latter reason was particularly important, because the company needs to manage resources in multiple AWS accounts and because it reduces the probability of human error.

There are additional security benefits with AWS Systems Manager, including:

VR Beneficios is also eliminating network management overhead, which includes eliminating the need to open inbound network ports. In traditional architectures, these ports must be maintained at multiple layers, including network firewalls, or in some cases, direct public access for systems directly connected to the Internet.  Session Manager allows us to remove the need for our managed instances to be publicly accessible. Managed instances that are managed with Session Manager can also make use of AWS PrivateLink, which restricts traffic between EC2 managed instances and AWS Systems Manager to the Amazon network.

Additional benefits that VR Beneficios plans to use in the future include limiting managed instance access via resource tags and instance IDs, the “Run As” capability to restrict the level of access users can assume when using Session Manager, and also using SCP-based file transfers, as needed.

Architecture
VR Beneficios uses the workflow shown in the following diagram to manage on-premises and EC2 instances with Session Manager. It consists of multiple AWS accounts to manage the development, testing, and production environments, along with an on-premises environment.

There is a centralized management account where all the administrator accounts live. This configuration allows for all the management users to be in a single account, along with the ability to use customized IAM policies to include access to S3 and CloudWatch Logs.

 

VR Beneficios completed the rollout of Systems Manager to manage both AWS Cloud and on-premises resources, including hundreds of resources managed by Systems Manager. As part of the rollout, all VPCs were configured with SSM endpoints to ensure that all traffic remains local within the AWS infrastructure.

Break/fix scenarios
VR Beneficios uses interactive shell access mainly in break/fix scenarios when DevOps automation is not an option. The following scenario shows how this would happen:

  1. Go to the AWS CLI and connect to the instances using the Session Manager Plugin.
  2. Always make connections from the Management account using the role-arn of the other accounts to switch. Examples of access:
    – aws ssm start-session –target “i-XXXXXXXXXXXX” –profile vrdev
    – aws ssm start-session –target “i-XXXXXXXXXXXX ” –profile vrtest
    – aws ssm start-session –target “i-XXXXXXXXXXXX ” –profile vrprd
    – aws ssm start-session –target “i-XXXXXXXXXXXX ” –profile vrbeneficios
  3. Because the audit capability of Session Manager is a huge benefit, there are also periodic reviews of the activity captured via CloudWatch Logs and the S3 bucket configured for session command output.

Remote infrastructure management
Although this post focuses on the company’s use of Session Manager, VR Beneficios also uses Systems Manager to manage its infrastructure remotely. This includes using the Run Command to deploy the CloudWatch agent across all environments to keep all AWS-based agents up to date and also gather inventory data.

Summary
This solution described in this post using Session Manager is just one of the many ways that VR Beneficios leverages AWS management and governance services. Using these tools, the company maintains control over cost, compliance, and security without impacting its pace of innovation and operational efficiency.

About the Author

Cesar Soares is a DevOps and cloud infrastructure manager at VR Beneficios, with over 17 years of experience in the technology field. He actively works with AWS and AWS Premier Partners to continue pushing the pace of innovation. At the same time, he seeks increasing operational efficiency and security across multiple environments, including offshore and nearshore operations, and the AWS platform. Cesar is active in the technology community and can be reached at https://www.linkedin.com/in/alexandrecesarsoares.