Secure Certificate Management with HashiCorp Consul on AWS

By Peter McCarron, Sr. Manager, Partner Marketing at HashiCorp
By Paul Banks, Sr. Engineer, Consul at HashiCorp

The adoption of service mesh has been accelerated by the emergence of microservices.

Organizations leverage orchestration tools, like HashiCorp Nomad or Amazon Elastic Kubernetes Service (Amazon EKS) for deploying services, and then use a service mesh such as HashiCorp Consul or AWS App Mesh to manage the connections between them.

These service mesh solutions provide a number of important capabilities, such as traffic management, secure connections with mTLS (mutual TLS authentication), and load balancing for managing services at scale.

When considering service mesh, a lot of focus goes into discussing connection automation. However, another area of great importance is in how public key infrastructure (PKI) is managed in these environments.

In this post, we’ll cover how HashiCorp Consul and the AWS Certificate Manager (ACM) Private CA are able to automate certificate management, perform signing operations, and rotate private certificates as needed.

HashiCorp is an AWS Partner Network (APN) Advanced Technology Partner with AWS Competencies in Security, DevOps, and Containers. Enterprise versions of HashiCorp products enhance the open source tools with features that promote collaboration, operations, governance, and multi-data center functionality.

What is ACM Private CA?

Before diving into Consul’s capabilities, it’s important to first understand what ACM Private CA is and how it can help protect your network against unauthorized connections.

ACM Private CA is an extension of the AWS Certificate Manager (ACM), AWS’s tool for helping manage public SSL/TLS certificates. Organizations leverage ACM for the distribution of these certificates, allowing customers to access the domains that they manage.

Inversely, internal networks or private networks require private certificates that are more strictly regulated inside an organization. These private certificates are critical for controlling which users and devices have the ability to establish encrypted communications with internal business applications and systems.

ACM Private CA is designed to help organizations manage these sensitive certificates and add additional capabilities, like creating hierarchies and leveraging tools such as Consul for distribution and rotation.

How Consul Utilizes Certificate Authorities

HashiCorp Consul’s service mesh capabilities are designed to automate network connections between services in any environment. As part of this automation, Consul has the ability to provide a new service with a certificate from a certificate authority (CA) and refresh certificates for existing services.

At present, Consul supports three different CA providers:

If a provider has not been specified, then Consul will automatically select itself as the CA provider and generate a root certificate and private key as part of the bootstrap process.

Initializing the ACM Private CA Provider

As of Consul 1.7, ACM Private CA has been added to the pre-installed CA providers that Consul supports. This means that in order to start using the ACM Private CA, all users have to do is specify it in their initial configuration.

Here’s an example of what this would look like:

connect {
    enabled = true
    ca_provider = "aws-pca"
    ca_config {

	region = “region”
	root_arn = “RootARN”
	Intermediate_arn =”IntermediateTemplateARN”
    }
}

As stated earlier, the selection of a CA provider happens when a new Consul leader is elected in each data center. In order to ensure Consul knows to select ACM Private CA as the provider, users must specify it in the `ca_provider` field within the `connect` stanza of their Consul configuration file.

Once initialized, this configuration will create a new certificate authority in ACM Private CA to act as the CA for Consul Connect. It will need AWS Identity and Access Management (IAM) privileges to do this and set the default region from an instance role in Amazon Elastic Compute Cloud (Amazon EC2), a credential file or environment variables.

Optionally, an existing Root CA’s ARN can be configured to be used rather than creating a new one. Consul will automate creating subordinate CAs in each secondary data center, and having those signed by the root in the primary data center.

Viewing Root Certificate and Provider Configuration

Users can interact with Consul via a number of API endpoints. To view the root certificate, users can leverage the List CA roots endpoint. As new root certificates are created as part of a rotation process, this endpoint will list previous root certificate values as well.

Currently, the Consul PCA provider is not able to cross-sign root certificates from another CA. This could mean some new TLS connections will fail validation for a short time and need to be retried while the CA change is propagated through the cluster, which typically takes a few seconds.

Over time, users may need to reconfigure some of the parameters of the PCA provider. For example, let’s say the security team has issued a new organizational policy that requires all certificates issued must have a TTL of 48 hours.

First, users can get a quick view of the current configuration of the provider through the Get CA Configuration API endpoint:

curl http://localhost:8500/v1/connect/ca/configuration
{
   "Provider": "aws-pca",
    "Config": {
        "LeafCertTTL": "72h",
        "RotationPeriod": "2160h"
    },
    "CreateIndex": 5,
    "ModifyIndex": 5
}

The returned values from above may not be the complete configuration; Consul may scrub the returned data for sensitive information.

Based on the above results, we need to reduce the TTL of the Leaf Certificates. Fortunately, this can be done using the Update CA Configuration API endpoint.

First, save the updated configuration as a payload file:

{
    "Provider": "aws-pca",
    "Config": {
        "LeafCertTTL": "48h",
 “Root_Arn”: “RootArn”
        "RotationPeriod": "2160h"
    }
}

Next, deliver the file via the ‘set-config’ command:

consul connect ca set-config -config-file=”payload.json”

Conclusion

What does this all mean for users? Organizations that leverage Consul and the ACM Private CA provider can automate the signing and rotation of certifications for their applications.

In addition to reducing the pain of certificate rotation events through automation, leveraging these tools also reduces the risk of organizations relying on long-lived certificates.

Given the length of time needed to manually update business applications with the proper certificates, it makes sense that many organizations would look to reduce the number of times they have to make these rotations.

Moving to microservices does not make this challenge any easier; if anything, it increases the number of services that require updating during rotation events and expands the attack surface.

Improving security posture with short-lived certificates is another reason organizations making this transition should leverage service mesh tools like HashiCorp Consul.

For more information on Consul, please visit consul.io >>

The content and opinions in this blog are those of the third party author and AWS is not responsible for the content or accuracy of this post.

.

.

HashiCorp – APN Partner Spotlight

HashiCorp is an AWS Security Competency Partner. Enterprise versions of products like Consul enhance the open source tools with features that promote collaboration, operations, governance, and multi-data center functionality.

Contact HashiCorp | Solution Overview

*Already worked with HashiCorp? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.

AWS Partner Network (APN) Blog