Securing your bastion hosts with Amazon EC2 Instance Connect

In a previous blog post, I discussed how you can use AWS Systems Manager Session Manager to securely connect to your private instances in your virtual private cloud (VPC) without needing an intermediary bastion host, open ports, or a key pair assigned to the instances. In this post, I cover how you can improve the security of your existing bastion hosts by using Amazon Elastic Compute Cloud (Amazon EC2) Instance Connect. I also demonstrate how you can use an AWS Lambda function to automate your security group configuration to allow access from the published IP address range of the EC2 Instance Connect service. This is necessary if you want to connect to your instances using Instance Connect from the Amazon EC2 console.

Traditionally, the Amazon EC2 bastion host instance is associated with only one key pair for secure access. To allow multiple individuals access to the bastion host, you either have to share the key pair or add public keys provided by the individuals to authorized keys on the bastion host, adding a management overhead of ensuring that the list of authorized keys is kept up-to-date.

With EC2 Instance Connect, you no longer have to associate a key pair to the instance and do not have to permanently add user keys to authorized keys. Rather, you can now push keys for the short term and restrict access using familiar AWS Identity and Access Management (IAM) policies.

Before you start

For details on Amazon EC2 Instance Connect, see the Connect Using EC2 Instance Connect documentation.

In this blog post, you’ll be using the ec2-instance-connect command in the AWS Command Line Interface (AWS CLI) If you don’t see this command, please upgrade the AWS CLI, before continuing.

Setup

To enable connections using EC2 Instance Connect, you need to have an EC2 Instance Connect package installed on your instance. You can do so by using the yum install command.

[ec2-user ~]$ sudo yum install -y ec2-instance-connect

If your IAM users or roles are only using the Amazon EC2 console or a Secure Shell (SSH) client to connect to the instance, use the following policy. You can attach this policy to existing users or roles to allow them to use this feature.

EC2InstanceConnectSSHPolicy:
  Type: AWS::IAM::ManagedPolicy
  Properties:
    Description: Policy to use SSH client to connect to an instance using EC2 Instance Connect
	PolicyDocument:
	  Version: "2012-10-17"
	  Statement:
	    - Action:
		  - ec2-instance-connect:SendSSHPublicKey          
		  Effect: Allow
		  Resource:            
		  Fn::Sub:arn:aws:ec2:${AWS::Region}:${AWS::AccountId}:instance/${BastionInstance}  
		  Condition:            
		    StringEquals:              
			  ec2:osuser: "ec2-user

If you would also like to allow EC2 Instance Connect CLI to connect to the instance, add a statement in the policy document that allows an ec2:DescribeInstances action.

- Action:
- ec2:DescribeInstances
Effect: Allow
Resource: "*"

Testing

For testing, I’ve provided an AWS CloudFormation template that sets up the following environment.

A VPC, using the AWS VPC Quick Start, with two public subnets spanning two Availability Zones.
In one of the public subnets, an Amazon Linux 2 demo instance with the latest EC2 Instance Connect installed. Although this instance doesn’t have any key pair associated with it, it can be accessed securely using your private key or the EC2 Instance Connect CLI.
An AWS Lambda function that parses the published IP ranges and extracts the IP range for the EC2_INSTANCE_CONNECT service in the region.
A security group that allows access to port 22 from your IP range as well as the published EC2 Instance Connect IP address range for the AWS Region.
Two customer managed policies, which can be attached to existing users or roles to allow connecting to EC2 instances using EC2 Instance Connect:
- EC2InstanceConnectSSHPolicy – to allow connection using the Amazon EC2 console and an SSH client.
- EC2InstanceConnectCLIPolicy – to allow connection using the Amazon EC2 console, an SSH client, and the EC2 Instance Connect CLI.
An IAM Role EC2InstanceConnectRole to test the connection using EC2 Instance Connect. This role is for illustration only. In the following test steps, I’ll assume that your terminal session has the right privileges to invoke EC2 Instance Connect. If you would like to use this role, you can run the following command and use the command output to set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables.

aws sts assume-role --role-arn <EC2InstanceConnectRoleARN> --role-session-name ec2-ic-session

Deployment steps

Use this launch link to deploy the provided AWS CloudFormation template in the AWS Ohio Region (us-east-2) using the AWS CloudFormation console. You will need to choose two Availability Zones for the AvailabilityZones parameter and provide your CIDR IP range permitted to access the bastion hosts.
Once the CloudFormation stack reaches CREATE_COMPLETE status, go to the Outputs tab and note down the values of BastionInstance, BastionDNSName, and DeployAZ. For ease of testing, save these outputs as environment variables in a terminal.

export BASTION_INSTANCE=<Output of BastionInstance>

export BASTION_DNS_NAME=<Output of BastionDNSName>

export DEPLOY_AZ=<Output of DeployAZ>

export AWS_DEFAULT_REGION=<Your deployment Region, e.g., us-east-2>

Start a session

There are three ways to start a session: using an SSH client, using the EC2 Instance Connect CLI, or using the Amazon EC2 console.

Start a session using an SSH client

If you have an existing key (e.g., ~/.ssh/id_rsa), you can use that. Otherwise, create a new key with the following command:

ssh-keygen -t rsa -f my_rsa_key

Push your SSH public key to the instance.

aws ec2-instance-connect send-ssh-public-key --instance-id $BASTION_INSTANCE --availability-zone $DEPLOY_AZ --instance-os-user ec2-user --ssh-public-key file:///path/to/my_rsa_key.pub

Connect to the instance using your private key.

ssh -i /path/to/my_rsa_key ec2-user@$BASTION_DNS_NAME

Start a session using the EC2 Instance Connect CLI

If you don’t have the EC2 Instance Connect CLI installed, follow the instructions to install it. Then, use the mssh command and the instance ID to connect to the instance.

mssh $BASTION_INSTANCE

Start a session using the Amazon EC2 console

The AWS CloudFormation template automatically adds the IP address range of the EC2 Instance Connect service to the security group attached to the bastion instance.

To connect to the instance using the Amazon EC2 console:

Choose the instance, and then from the Actions menu, choose Connect.
In the options popup, choose I would like to connect with EC2 Instance Connect, and then choose Connect again.

A new browser window will open, and you’ll be connected to the instance.

Clean up

After you are done testing, go to the AWS CloudFormation console and delete the EC2-Instance-Connect-demo stack to avoid incurring charges for any resources created by the template.

Conclusion

I’ve demonstrated a simple way of adding an IAM policy to your existing IAM users or roles to allow securely connecting to your bastion hosts, without the need of a key pair or long-term authorization of user keys.

With this approach, you no longer have to associate a key pair to your instance, and only users who are authorized to invoke EC2 Instance Connect commands can initiate a session. This provides an improved security and audit posture. In addition, the post demonstrates how you can use a simple Lambda function to automate adding EC2 Instance Connect IP address ranges to the instance, if you want to allow connection using the Amazon EC2 console.

I hope you enjoyed reading this post. If you have any feedback on the steps in this post, please add your comments.

Integration & Automation