How to run AWS CloudHSM workloads on AWS Lambda

November 21, 2021: CloudHSM SDK3 does not support serverless environments, and we strongly recommend deploying SDK5. We are currently working on an updated blog post.

AWS CloudHSM is a cloud-based hardware security module (HSM) that enables you to generate and use your own encryption keys on the AWS Cloud. With CloudHSM, you can manage your own encryption keys using FIPS 140-2 Level 3 validated HSMs. CloudHSM also automatically manages synchronization, high availability and failover within a cluster.

When the service first launched, many customers ran CloudHSM workloads on Amazon Elastic Compute Cloud (Amazon EC2), which required the CloudHSM client to be installed on the Amazon EC2 instance in order to communicate with the CloudHSM cluster. Today, we see customers who are interested in leveraging CloudHSM for serverless workloads using AWS Lambda, but when using Lambda there is no “instance” to install the CloudHSM client on. This blog post shows a workaround that can be used to satisfy the CloudHSM client installation requirement on Lambda functions to be able to run CloudHSM workloads within these Lambda functions.

The workaround is performed by first packaging the CloudHSM client and its requirements in a Lambda layer, and then running the CloudHSM client in a child process from within the Lambda function code to allow communication with the HSMs in your CloudHSM cluster. By leveraging this approach, you gain the benefits of serverless computing (such as increased scalability and decreased admin overhead), as well as the ability to integrate with other AWS services like Amazon CloudWatch Events, Amazon Simple Storage Service (Amazon S3) and AWS Config.

Why would I want to run CloudHSM workloads on Lambda?

Below are some specific use cases enabled by this solution:

When a file is added to an Amazon S3 bucket, you can trigger a Lambda function to encrypt or decrypt the file using keys stored in CloudHSM.
When a file is added to an Amazon S3 bucket, you can trigger a Lambda function to create a digital signature for the file using a private key stored in CloudHSM. This digital signature can then be used to ensure file integrity.
You can create a custom AWS Config rule that checks to ensure files in a directory or a bucket have not been tampered with by verifying their digital signatures using keys stored in CloudHSM.

Solution overview

This solution shows you how to package the CloudHSM client binary and its dependencies (configuration files and libraries) as well as the CloudHSM Java JCE library to a Lambda layer which is attached to the Lambda function. This enables the function to run the CloudHSM client daemon in the background as a child process, allowing it to connect to the CloudHSM cluster and to perform cryptographic tasks such as encryption and decryption operations.

Using a Lambda layer decouples the code of the Lambda function from the CloudHSM client and the CloudHSM Java JCE library. This way, when a new version of the CloudHSM client and the CloudHSM Java JCE library is released, it can be included in a new Lambda layer version and attached to the Lambda function without needing to rebuild the Lambda function package.

The example solution below includes a complete Java sample for the Lambda function. It uses the CloudHSM Java JCE library to generate a symmetric key on the HSM, and it uses this key to encrypt and decrypt after starting the CloudHSM client. Maven (a build automation tool) will be used to build the Lambda function package.

The solution uses AWS Secrets Manager to store and retrieve the crypto user (CU) credentials that are needed to perform cryptographic operations. If the HSM IPs of the CloudHSM cluster are changed (for example, if the HSMs are deleted and re-created), the Lambda function will automatically update the configuration during runtime.

Note:

The solution only works with version 2.0.4 or later of the CloudHSM client and CloudHSM Java JCE library.

In this workaround, the client is started at the beginning of each Lambda invocation, and is stopped at the end of the invocation. Due to the way Lambda works, the client can’t persist through multiple invocations.

Secrets Manager uses AWS Key Management Service to secure its data. If your workload requires that all data be secured using HSMs under your sole control, without reliance on IAM credentials, this solution may not be appropriate. You should work with your security or compliance officer to ensure you’re using a method of securing HSM login credentials that meets your application and security needs.

Prerequisites

Figure 1: Architectural diagram

Here are the resources you’ll need in order to follow along with the example in Figure 1:

An Amazon Virtual Private Cloud (Amazon VPC) with the following components:
1. Private subnets in multiple Availability Zones to be used for the HSM’s elastic network interfaces (ENIs).
2. A public subnet that contains a network address translation (NAT) gateway.
3. A private subnet with a route table that routes internet traffic (0.0.0.0/0) to the NAT gateway. You’ll use this subnet to run the Lambda function. The NAT gateway allows you to connect to the CloudHSM, CloudWatch Logs and Secrets Manager endpoints.
Note: For high availability, you can add multiple instances of the public and private subnets mentioned in Prerequisites 1.b and 1.c. For more information about how to create an Amazon VPC with public and private subnets as well as a NAT gateway, refer to the Amazon VPC user guide.
An active CloudHSM cluster with at least one active HSM. The HSMs should be created in the private subnets mentioned in Prerequisite 1.a. You can follow the Getting Started with AWS CloudHSM guide to create and initialize the CloudHSM cluster.
An Amazon Linux 2 EC2 instance with the CloudHSM client installed and configured to connect to the CloudHSM cluster. The client instance should be launched in the public subnet mentioned in Prerequisite 1.b. You can again refer to Getting Started With AWS CloudHSM to configure and connect the client instance.

Note: You only need the client instance to build the Lambda function package. You can terminate the instance after the package has been created.
CU credentials. You can create a CU by following the steps in the user guide.

A server/machine with AWS Command Line Interface (AWS CLI) installed and configured. You’ll need this to follow along, as the example uses AWS CLI to create and configure the necessary AWS resources. The IAM user/role should have at minimum the permissions in the below policy attached to it to follow this example. Make sure you replace the <REGION> and <ACCOUNT-ID> tags below with the actual Region and account ID you are using.


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "secretsmanager:CreateSecret",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "secretsmanager:Name": "CloudHSM_CU"
                }
            }
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupEgress",
                "lambda:CreateFunction",
                "lambda:InvokeFunction",
                "lambda:GetLayerVersion",
                "lambda:PublishLayerVersion",
                "iam:GetRole",
                "iam:CreateRole",
                "iam:AttachRolePolicy",
                "iam:PutRolePolicy",
                "iam:PassRole",
                "secretsmanager:DescribeSecret",
                "secretsmanager:GetResourcePolicy",
                "secretsmanager:GetSecretValue",
                "secretsmanager:PutResourcePolicy",
                "logs:FilterLogEvents"
            ],
            "Resource": [
                "arn:aws:ec2:<REGION>:<ACCOUNT-ID>:security-group/outbound-443",
                "arn:aws:lambda:<REGION>:<ACCOUNT-ID>:function:cloudhsm_lambda_example",
                "arn:aws:lambda:<REGION>:<ACCOUNT-ID>:layer:cloudhsm-client-layer",
                "arn:aws:lambda:<REGION>:<ACCOUNT-ID>:layer:cloudhsm-client-layer:*",
                "arn:aws:iam::<ACCOUNT-ID>:role/cloudhsm_lambda_example_role",
                "arn:aws:secretsmanager:<REGION>:<ACCOUNT-ID>:secret:CloudHSM_CU*",
                "arn:aws:logs:<REGION>:<ACCOUNT-ID>:log-group:/aws/lambda/cloudhsm_lambda_example:log-stream:"
            ]
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeVpcs",
                "ec2:CreateSecurityGroup",
                "ec2:DescribeSubnets",
                "cloudhsm:DescribeClusters",
                "ec2:DescribeSecurityGroups",
                "ec2:AuthorizeSecurityGroupEgress"
            ],
            "Resource": "*"
        }
    ]
}

Step 1: Build the Lambda function package

In this step, you’ll build the Lambda function package using Maven. For more information about using Maven to build an AWS Lambda Java package, refer to the AWS Lambda developer guide.

On your CloudHSM client instance, install the CloudHSM Java JCE library by following the steps in the user guide.
Install OpenJDK 8 and Maven:
```
$ sudo yum install -y java maven
	
```
Download the sample code, unzip it and move to the created directory. The directory will have the name aws-cloudhsm-on-aws-lambda-sample-master and will include:
- A file with the name pom.xml that contains the Maven project configuration.
- A file with the name SymmetricKeys.java which is also available on the AWS CloudHSM Java JCE samples repo. This file contains the function that you’ll use to generate the advanced encryption standard (AES) key.
- A file with the name AESGCMEncryptDecryptLambda.java, which will run when the Lambda function is invoked:
```
$ wget https://github.com/aws-samples/aws-cloudhsm-on-aws-lambda-sample/archive/master.zip
$ unzip master.zip
$ cd aws-cloudhsm-on-aws-lambda-sample-master/
	
```

Create a Java Archive (JAR) package by running the below commands. This will create the JAR file under the target/ directory with the name cloudhsm_lambda_project-1.0-SNAPSHOT.jar.


$ export CLOUDHSM_VER=$(ls /opt/cloudhsm/java/ | grep "cloudhsm-[0-9\.]\+.jar" | grep -o "[0-9\.]\+[0-9]")
$ export LOG4JCORE_VER=$(ls /opt/cloudhsm/java/ | grep "log4j-core-[0-9\.]\+.jar" | grep -o "[0-9\.]\+[0-9]")
$ export LOG4JAPI_VER=$(ls /opt/cloudhsm/java/ | grep "log4j-api-[0-9\.]\+.jar" | grep -o "[0-9\.]\+[0-9]")
$ mvn validate && mvn clean package

Step 2: Create the Lambda layer

In this step, you’ll create the Lambda layer that contains the CloudHSM client and its dependencies and the CloudHSM Java library JARs.

On your CloudHSM client instance, create a directory called “layer” and change directories to it:
```
$ mkdir ~/layer && cd ~/layer
	
```
Create the following directories, which you’ll use in the next steps to hold the CloudHSM binary and its prerequisites such as configuration files and libraries, and the CloudHSM Java JCE JARs:
```
$ mkdir -p lib cloudhsm/bin cloudhsm/etc java/lib
	
```

Copy the cloudhsm_client binary and the needed configuration files to the directories you created in the previous step.


$ cp /opt/cloudhsm/bin/cloudhsm_client cloudhsm/bin
$ cp -r /opt/cloudhsm/etc/{cloudhsm_client.cfg,customerCA.crt,client.crt,client.key,certs} cloudhsm/etc

Add the necessary libraries by running the commands below. These libraries are needed by the Lambda function to be able to run the cloudhsm_client binary.
```
$ cp /opt/cloudhsm/lib/libcaviumjca.so lib/
$ ldd /opt/cloudhsm/bin/cloudhsm_client | awk '{print $3}' | grep "^/" | xargs -I{} cp {} lib/
	
```
Add the CloudHSM Java JCE Jars by running the commands below. These JARs include the classes needed by the Lambda function code to run.
```
$ cp /opt/cloudhsm/java/{cloudhsm-[0-9]*.jar,log4j-*-*.jar} java/lib/
	
```
Create the Lambda layer ZIP archive by running the command below. This will create the archive with the name layer.zip in the home directory.
```
$ zip -r ~/layer.zip * 
	
```
Move the ZIP archive (layer.zip) to the server/machine with AWS CLI installed and configured, and run the below command to create the Lambda layer with the name cloudhsm-client-layer.
```
$ aws lambda publish-layer-version --layer-name cloudhsm-client-layer --zip-file fileb://layer.zip --compatible-runtimes java8
	
```

Step 3: Create a secret to store the CU credentials

In this step, you will use Secrets Manager to create a secret to store your CU credentials. You must perform this step on your server/machine that has AWS CLI installed and configured.

Run the following command to create a secret with the name CloudHSM_CU that contains your CU user name and password (Prerequisite 4). Make sure to replace the user name and password below with your actual CU user name and password.


$ export HSM_USER=<user>
$ export HSM_PASSWORD=<password>
$ aws secretsmanager create-secret --name CloudHSM_CU --secret-string "{ \"HSM_USER\": \"$HSM_USER\", \"HSM_PASSWORD\": \"$HSM_PASSWORD\"}"

Step 4: Create an IAM role for the Lambda function

In this step, you’ll create an IAM role that has the permissions necessary for it to be assumed by the Lambda function.

On the server/machine with AWS CLI installed and configured, create a new file with the name trust.json.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Create a role named cloudhsm_lambda_example_role using the following AWS CLI command:


$ aws iam create-role --role-name cloudhsm_lambda_example_role --assume-role-policy-document file://trust.json

Run the commands below to create a new file named policy.json. The policy in this file allows the IAM role to perform the following actions:

Writing to CloudWatch Logs. This permission allows the IAM role to write to the CloudWatch Logs of the Lambda function. You can then use the logs for troubleshooting. For more information about accessing CloudWatch Logs for Lambda, refer to this guide.
Retrieving the CU secret value from Secrets Manager. The CU credentials stored in the CU secret are needed by the Lambda function to be able to log-in to the CloudHSM cluster.
Describing CloudHSM clusters. This permission allows the Lambda function to check the current HSM IPs and update its configuration if the IPs have changed.


$ export SECRET_ARN=$(aws secretsmanager describe-secret --secret-id "CloudHSM_CU" --query "ARN" --output text)

$ cat <<EOF> policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CWLogs",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        },
        {
            "Sid": "SecretsManager",
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "$SECRET_ARN"
        },
        {
            "Sid": "CloudHSM",
            "Effect": "Allow",
            "Action": "cloudhsm:DescribeClusters",
            "Resource": "*"
        }
    ]
}
EOF

Attach the policy to the IAM role created in step 2 of this section by running the following command:


$ aws iam put-role-policy --role-name cloudhsm_lambda_example_role --policy-name cloudhsm_lambda_example_policy --policy-document file://policy.json

Attach the AWS managed policy AWSLambdaVPCAccessExecutionRole to the created role by running the command below. This policy allows the IAM role to access the VPC, which is necessary in order to run the Lambda function in a VPC and a subnet.
```
$ aws iam attach-role-policy --role-name cloudhsm_lambda_example_role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
	
```

To make sure the CU secret is only accessible to the Lambda function role, run the below commands to attach a resource-based policy to the secret:


$ export ROLE_ARN=$(aws iam get-role --role-name cloudhsm_lambda_example_role --query Role.Arn --output text)
$ export ASSUMED_ROLE_ARN=$(echo $ROLE_ARN | sed -e "s/:iam:/:sts:/" -e "s/:role/:assumed-role/" -e "s/$/\/cloudhsm_lambda_example/")
$ export ROOT_ARN=$(echo $ROLE_ARN | sed "s/:role.*/:root/")
$ cat <<EOF> sm_policy.json
{ "Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Deny",
			"Action": "secretsmanager:GetSecretValue",
			"NotPrincipal": {"AWS": [
				"$ASSUMED_ROLE_ARN",
				"$ROLE_ARN",
				"$ROOT_ARN"
			]},
				"Resource": "*"
		}
	]
}
EOF

$ aws secretsmanager put-resource-policy --resource-policy file://sm_policy.json --secret-id CloudHSM_CU

Step 5: Create the Lambda function

In this step, you will create a Lambda function with the necessary settings.

On the server/machine with AWS CLI installed and configured, run the command below to create a security group with the name outbound-443. This security group will be attached to the Lambda function to allow it to connect to the CloudWatch Logs, Secrets Manager and CloudHSM endpoints. Make sure to replace the CLUSTER_ID below with the actual CloudHSM cluster ID of your environment.


$ export CLUSTER_ID=<cluster-xxxxxxxxxx>
$ export CLUSTER_VPC=$(aws cloudhsmv2 describe-clusters --filters clusterIds=$CLUSTER_ID --query Clusters[0].VpcId --output text)
$ export OUTBOUND_SG=$(aws ec2 create-security-group --group-name outbound-443 --description "Allow outbound access to port 443" --vpc-id $CLUSTER_VPC --output text)
$ aws ec2 authorize-security-group-egress --group-id $OUTBOUND_SG --protocol tcp --port 443 --cidr 0.0.0.0/0

Move the JAR package generated in step 4 of the Step 1 section to the current directory on the server/machine that has AWS CLI installed and configured (The file was generated on the CloudHSM client instance under ~/aws-cloudhsm-on-aws-lambda-sample-master/target/cloudhsm_lambda_project-1.0-SNAPSHOT.jar).

Replace the cluster ID and subnet ID below with the CloudHSM cluster ID of your environment, and the ID of the private Lambda subnet in your environment (Prerequisite 1.c), then run the commands below. These commands set environment variables that you’ll need for the next command.


$ export CLUSTER_ID=<cluster-xxxxxxxxxx>
$ export SUBNET_ID=<subnet-xxxxxxxx>
$ export CLUSTER_VPC=$(aws cloudhsmv2 describe-clusters --filters clusterIds=$CLUSTER_ID --query Clusters[0].VpcId --output text)
$ export OUTBOUND_SG=$(aws ec2 describe-security-groups --filters Name=group-name,Values=outbound-443  --query SecurityGroups[0].GroupId --output text)
$ export CLUSTER_SG=$(aws cloudhsmv2 describe-clusters --filters clusterIds=$CLUSTER_ID --query Clusters[0].SecurityGroup --output text)
$ export ROLE_ARN=$(aws iam get-role --role-name cloudhsm_lambda_example_role --query Role.Arn --output text)
$ export LAYER_ARN=$(aws lambda get-layer-version --layer-name cloudhsm-client-layer --version-number 1 --query LayerVersionArn --output text)

Create a Lambda function with the name cloudhsm_lambda_example by running the below command:


$ aws lambda create-function --function-name "cloudhsm_lambda_example" \
--runtime java8 \
--role $ROLE_ARN \
--handler "com.amazonaws.cloudhsm.examples.AESGCMEncryptDecryptLambda::myhandler" \
--timeout 600 \
--memory-size 512 \
--vpc-config SubnetIds=$SUBNET_ID,SecurityGroupIds=$CLUSTER_SG,$OUTBOUND_SG \
--environment "Variables={CLUSTER_ID=$CLUSTER_ID, SECRET_ID=CloudHSM_CU,liquidsecurity_daemon_id=1}" \
--layers $LAYER_ARN \
--zip-file fileb://cloudhsm_lambda_project-1.0-SNAPSHOT.jar

The command will create a Lambda function with the following configuration:

Runtime: Java8
Execution Role: The role you created in the Step 4 section.
Handler: The name of the class and the function in the package created in the Step 1 section.
Timeout: 10 minutes.
Memory size: 512 MB.
Subnet: The private Lambda subnet in your environment (Prerequisite 1.c).
Security Groups: The CloudHSM cluster security group AND the security group created in step 1 of the Step 5 section for outbound access to port 443 (outbound-443).
Code/Package: The JAR package you created in step 4 of the Step 1 section.
Layer: The layer created in the Step 2 section.
Environmental Variables:
- CLUSTER_ID = the CloudHSM cluster ID in your environment
- SECRET_ID = the ID of the secret you created in the Step 3 section
- liquidsecurity_daemon_id = 1 (this is needed by the cloudhsm_client binary)

Step 6: Run the Lambda function

In this step, you will invoke the Lambda function and check the logs to view the output.

You can invoke the Lambda function using the following command. This will execute the code in the package you created in Step 1.
```
$ aws lambda invoke --function-name cloudhsm_lambda_example out.txt
	
```

You can check the function’s CloudWatch Log group with a command like this one:


$ aws logs filter-log-events --log-group-name "/aws/lambda/cloudhsm_lambda_example" --start-time "`date -d "now -5min" +%s`000" --query events[*].message --output text | sed "s/\t/\n/g"

If the Lambda function was successful, the output of the function should look something like the example below:


START RequestId: 39c627f2-3908-4424-97ef-038c28a72f9a Version: $LATEST

* Running GetSecretValue to get the CU credentials ...
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

SLF4J: Defaulting to no-operation (NOP) logger implementation

SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

* Running DescribeClusters to get the HSM IP ...
DescribeClusters returned the HSM IP = 1.2.3.4
* Getting the HSM IP inf the configuration file ...
The configuration file has the HSM IP = 1.2.3.4
* Starting the cloudhsm client ...
* Waiting for the cloudhsm client to start ...
* cloudhsm client started ...
* Adding the Cavium provider ...
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

* Using credentials to Login to the CloudHSM Cluster ...
Login successful!
* Generating AES Key ...
* Generating Random data to encrypt ...
Plain Text data = 3B0566E9A3FADA8FED7D6C88FE92ECBE8526922E84489AB48F1F3F3116235E69
* Encrypting data ...
Cipher Text data = CA6D80AD34BBADEF34275743F309E6730ABC66BA19C2EADC731899B0FB86564EDDB9F7FC103E1C9C2A6A1E64BF2D2C48
* Decrypting ciphertext ...
Decrypted Text data = 3B0566E9A3FADA8FED7D6C88FE92ECBE8526922E84489AB48F1F3F3116235E69
 * Successful decryption
* Logging out the CloudHSM Cluster
* Closing client ...
END RequestId: 39c627f2-3908-4424-97ef-038c28a72f9a

REPORT RequestId: 39c627f2-3908-4424-97ef-038c28a72f9a
Duration: 11990.69 ms
Billed Duration: 12000 ms
Memory Size: 512 MB
Max Memory Used: 103 MB

Note: The StatusLogger No log4j2 configuration file found error above is normal and can be ignored. This is related to missing log4j configuration which is normally used to configure logging, but is not needed in this case as the log messages are being written to CloudWatch Logs by default.

Conclusion

This solution demonstrates how to run CloudHSM workloads on Lambda, which allows you to not only leverage the flexibility of serverless computing, but also helps you meet security and compliance requirements by performing cryptographic tasks such as encryption and decryption operations. This approach also allows you to integrate with other AWS services like Amazon CloudWatch Events, Amazon Simple Storage Service (Amazon S3), or AWS Config for a seamless experience across your environment.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS CloudHSM forum or contact AWS Support.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

AWS Security Blog