AWS OpsWorks for Puppet Enterprise and an alternate implementation for policy based auto signing

AWS OpsWorks for Puppet Enterprise was released in November of 2017. It has a secure API (associate node) that provides a secure, convenient, and AWS-integrated method to sign certificates for clients of OpsWorks for Puppet Enterprise. This secure API is ideal for use within a user data script when being used for AWS CloudFormation (which can be found here) or Auto Scaling Groups.

However, this blog post is about creating an alternate mechanism that allows end users to implement a different condition-checking mechanism before signing client certifications. This method is effective in environments such as proof of concept (PoC), development, or testing where clients aren’t executing the full AWS provisioning process. For example, this method could be used where customers build bare metal virtual machines (VMs) or containers. It could also be used where the VMs exist in a hybrid cloud environment or in an environment where OpsWorks for Puppet Enterprise is used to govern a customer’s private cloud VMs. It’s important to note that this method is not integrated with AWS CloudTrail, where every associate node API call would be tracked. When you use this alternative method, this tracking is not in effect.

Introduction

When a VM makes a request to sign in to the Puppet master, the master bases its approval of the client’s request on certain criteria. This is called policy-based auto signing. Due to the nature of Puppet and the possibility of exposing sensitive information, the signing of a client’s certificate must be approved carefully. The conceptual implementation is documented on the Puppet website.

This blog post provides one example of policy-based auto signing using the following assumptions:

The Puppet clients are Amazon EC2 instances.
These instances reside in the same VPC.
The Puppet master is equipped with required tools such as the AWS CLI and jq.
The Puppet master’s AWS client tool has been configured with sufficient authority to be able to access all the information on the EC2 instances, such as their private DNS names and their tags.

Set up the necessary components on the Puppet Enterprise master

(NOTE: a puppet module that can help with automating steps 5-8 is available. It can be found here and for reporting an issue about the module, you can visit this link. Please note that you have to edit common.yaml to enter your ID and secret access key. For usage help, please visit here.)

allow_unauthenticated_ca = true

Choose Add parameter.
Choose Commit.
Log in to the PE master using SSH and execute the following command to apply the configuration change:

puppet agent -tov

In the same SSH session on the PE master, ensure that the AWS CLI is configured with the proper Access Key ID and the Secret Access Key for the pe-puppet

sudo su - pe-puppet --shell /bin/bash
$ aws configure
AWS Access Key ID [None]: ************
AWS Secret Access Key [None]: **********************
Default region name [None]: us-east-1
Default output format [None]: json              
$ exit

NOTE: This is an important step for the Puppet Enterprise Puppet user (pe-puppet) because the policy-based signing script must be executable by the pe-puppet user.

Edit /etc/puppetlabs/puppet/puppet.conf file and ensure that following line appears under [main] section of the configuration file:

autosign = /opt/puppetlabs/autosign/autosign.sh

Create an auto sign structure and copy over the script. (You can find this script in Appendix A.) And restart Puppet Master Service:

mkdir -p /opt/puppetlabs/autosign
cp autosign.sh /opt/puppetlabs/autosign/autosign.sh
chown -R pe-puppet.pe-puppet /opt/puppetlabs/autosign
chmod 750 /opt/puppetlabs/autosign /opt/puppetlabs/autosign/autosign.sh   
/etc/init.d/pe-puppetserver restart

Now the Puppet master is ready to sign based on the policy. (For the explanation of the policy contained in the example script, see Appendix B.) On the Puppet client that has not been signed into a PE master, run the following command to request to be signed in.

puppet agent -tov --waitforcert=200

To confirm, you should see the following entries in the /var/log/puppetlabs/puppetserver/puppetserver.log file on the PE master:

2017-12-18 20:23:21,739 INFO  [qtp2115462635-102] [p.p.certificate-authority] Signed certificate request for ip-my-ip-address.ec2.internal
2017-12-18 20:23:22,636 INFO  [qtp2115462635-364] [puppetserver] mount[pe_packages] allowing * access
2017-12-18 20:23:22,637 INFO  [qtp2115462635-364] [puppetserver] mount[pe_modules] allowing * access

On the Puppet client’s command line, you should see something like the following:

[root@ip-my-ip-address ~]# puppet agent -tov --waitforcert=200
Info: Creating a new SSL key for ip-my-ip-address.ec2.internal
Info: Caching certificate for ca
Info: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for ip-my-ip-address.ec2.internal
Info: Certificate Request fingerprint (SHA256): 
FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF:FF [snip]
Info: Caching certificate for ip-my-ip-address.ec2.internal
Info: Caching certificate_revocation_list for ca
Info: Caching certificate for ca
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter]/ensure: created

Conclusion

That’s it. The Puppet Enterprise (PE) Master will sign the client certificates based on the policy you have defined in the autosign.sh. To recap, this blog post shows you how to create an alternate mechanism that allows end users to implement different condition-checking before signing client certificates. We walk you through an example of policy-based auto signing configuration and provide an example script.

To learn more about AWS OpsWorks for Puppet Enterprise, visit here.

Appendix A: autosign.sh

#!/bin/sh
 
# If we do not have aws cli or jq, exit with error immediately
which aws > /dev/null 2>&1 || exit 2
which jq > /dev/null 2>&1 || exit 3
 
usage()
{
  echo "$0 [private dns name of ec2 instance]"
  exit 1
}
 
[ $# -eq 1 ] || usage
 
clientname=$1
# Criteria 1: the puppet client must be visible in my VPC
[ `aws ec2 describe-instances --filters "Name=private-dns-name,Values=$clientname" --output=text | wc -l | sed 's/ //g'` -eq 0 ] && exit 1
 
# Criteria 2: the puppet client must have proper tags
UNIQUETAG="puppetclient"
INSTANCEID=`aws ec2 describe-instances --filters "Name=private-dns-name,Values=$clientname" --output=json | jq .Reservations[].Instances[].InstanceId`
[ "`aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCEID" --output json  | jq '.Tags[] | select(.Key=="myuniquetag").Value' | sed 's/"//g'`" != "$UNIQUETAG" ] && exit 1
 
# We passed everything, give it a thumbs up
exit 0

Appendix B: Policy description

The provided autosign.sh script uses two criteria to validate legitimacy of the puppet client’s certificate signing request.

The Amazon EC2 VM must exist in the same VPC. (This is verified using the aws ec2 describe-instances)
The unique tag created during the instantiation phase of the Amazon EC2 VM must exist and match what the Puppet master expects (configurable/adjustable in the sh script).

About the author

Andrew Park is a Cloud Infrastructure Architect at Amazon Web Services. Prior to AWS, Andrew has served 20+ years as a Linux Solution Engineer, a Linux infrastructure Architect/Administrator and a Cloud Engineer. He has been a key participating member of cloud adoption programs in many different Canadian banks, and he also has participated in many different open source projects like Samba, restricted shell (rssh), not red hat update (Yes, this was an actual project), Red Hat’s Anaconda and Agenda VR3. He works tirelessly on behalf of the clients in order that customers can enjoy economically friendly open source solutions. He is a foodie, loves to converse with people and machines alike, and a firm believer of Open Source philosophy.

AWS Cloud Operations Blog