Deploy private CAs with CloudFormation

TL;DR deploy this demo 🤓

If you ever find yourself need to programatically issue X.509 certificates to private servers without having to manage your own PKI infrstructure, you probably looked at the AWS option called Certificate Manager Private Certificate Authority. It was released circa 2018, but as of the time of writing (September 2019), it has not ascended to the level of CloudFormation support.

We've developed a template to roll out a subordinate CA and all the dependent resources, including CloudFront and S3 for CRL using CloudFormation, without any manual intervention. Although it is not currently possible to roll out a Root CA, since AWS haven't implemented a step to import the Root CA certificate in their API/SDK, this method works for a subordinate CA deployment with signing key securely kept in SSM.

Our solution uses the generic-custom-resource-provider as well as a helpful secret-provider resource to generate and store the RSA signing key in the AWS SSM Parameter Store, so that the key never leaves CFN/AWS.

Most of the secret sauce is in the pca.yml nested template, which implements this common flow:

• generate self-signed certificate using private key in SSM (GenerateSelfSignedCertV1)
• create a subordinate CA (PCASubordinateS1)
• retrieve the CSR (CSRSubordinateS1)
• sign the CSR using self-signed certificate (SignCSRSubordinateS1)
• import the signed certificate and activate the CA (ImportSubordinateCertificateS1)

Note: the S1 notation stands for Server CA v1

Since all the methods dealing with certficate data require bytes encoding, we can not just use the standard CFN/boto proxy implemented in generic-custom-resource-provider. Therefore we had to extend our resource provider with custom plugin architecture and then implemented our own handlers in acm_pca.yml (e.g.):

#!/usr/bin/env python

import os
import sys
import re
import random
import boto3
from OpenSSL import crypto
from OpenSSL import SSL
from datetime import datetime
from base64 import b64decode

class ACM_PCA:
    authorityKeyIdentifier = False

    def __init__(self, *args, **kwargs):
        self.verbose = bool(int(os.getenv('VERBOSE', 0)))
        if self.verbose: print(
            'args: {}, kwargs: {}'.format(args, kwargs),
            file=sys.stderr
        )

    def sign_csr(self, *args, **kwargs):
        if self.verbose: print(
            'args: {}, kwargs: {}'.format(args, kwargs),
            file=sys.stderr
        )
        private_key = self.load_private_key(
            self.get_private_key_pem(kwargs['PrivateKey'])
        )

        try:
            csr_pem = b64decode(kwargs['Csr']).decode()
        except:
            csr_pem = kwargs['Csr']
        try:
            csr_payload = re.search(
                '-----BEGIN CERTIFICATE REQUEST-----(.*)-----END CERTIFICATE REQUEST-----',
                csr_pem,
                re.IGNORECASE
            )
            assert csr_payload
            csr_formatted = '{}{}{}'.format(
                '-----BEGIN CERTIFICATE REQUEST-----',
                csr_payload.group(1).replace(' ', '\n').replace('\\n', '\n'),
                '-----END CERTIFICATE REQUEST-----'
            )
            csr_pem = csr_formatted
        except:
            pass

        csr = crypto.load_certificate_request(
            crypto.FILETYPE_PEM,
            csr_pem
        )

        if self.verbose: print(
            'csr: {}'.format(
                crypto.dump_certificate_request(crypto.FILETYPE_TEXT, csr).decode()
            ),
            file=sys.stderr
        )

        try:
            cert_pem = b64decode(kwargs['CACert']).decode()
        except:
            cert_pem = kwargs['CACert']
        ca_cert = crypto.load_certificate(
            crypto.FILETYPE_PEM,
            cert_pem
        )

        if self.verbose: print(
            'ca_cert: {}'.format(
                crypto.dump_certificate(crypto.FILETYPE_TEXT, ca_cert).decode()
            ),
            file=sys.stderr
        )

        cert = crypto.X509()
        cert.set_version(2)
        cert.set_serial_number(random.randint(
            1000000000000000000000000000000000000,
            9999999999999999999999999999999999999
        ))
        cert.set_issuer(ca_cert.get_subject())
        cert.set_subject(csr.get_subject())
        cert.set_pubkey(csr.get_pubkey())

        cert.add_extensions([
            crypto.X509Extension(
                b'basicConstraints',
                True,
                b'CA:TRUE'
            ),
            crypto.X509Extension(
                b'subjectKeyIdentifier',
                False,
                b'hash',
                subject=cert
            ),
            crypto.X509Extension(
                b'keyUsage',
                True,
                b'digitalSignature, cRLSign, keyCertSign'
            )
        ])

        if self.authorityKeyIdentifier:
            cert.add_extensions([
                crypto.X509Extension(
                    b'authorityKeyIdentifier',
                    False,
                    b'keyid:always,issuer',
                    issuer=ca_cert
                )
            ])

        cert.gmtime_adj_notBefore(0)
        cert.gmtime_adj_notAfter(int(kwargs['ValidityInSeconds']))
        cert.sign(private_key, kwargs['Digest'])

        if self.verbose: print(
            'cert: {}'.format(
                crypto.dump_certificate(crypto.FILETYPE_TEXT, cert).decode()
            ),
            file=sys.stderr
        )

        return {
            'Certificate': crypto.dump_certificate(crypto.FILETYPE_PEM, cert).decode()
        }
...

The CFN custom resource which calls the above method looks like this:

  SignCSRSubordinateS1:
    Type: 'Custom::SignCSRSubordinateS1'
    Condition: HasSubordinateCSR
    Version: 1.0
    Properties:
      ServiceToken: !Sub 'arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:generic-custom-resource-provider'
      Version: R1
      # custom module: acm_pca.py
      AgentService: acm_pca
      AgentType: custom
      AgentCreateMethod: sign_csr
      AgentCreateArgs: !Join
      - ''
      - - !Sub '{ "PrivateKey": "/rsa-private-keys/${NameTag}/key_pair",'
        - '"CACert": "'
        - Fn::Base64:
            !Sub '${GenerateSelfSignedCertV1.Certificate}'
        - '",'
        - '"Csr": "'
        - Fn::Base64:
            !Sub '${CSRSubordinateS1.Csr}'
        - '",'
        - '"ValidityInSeconds": 315360000,'
        - '"Digest": "sha256" }

To get out of \n escape hell, we base64 encode certificate and CSR data using internal CFN function before passing them on. In order to pass interger values, we need to specify AgentCreateArgs as properly formatted JSON, so since there is no short-hand notation for Fn::Base64 it looks a bit messy, but is functional.

Since the custom plugin requires access to crypto functions, we build all the dependencies in a Docker container using lambci/lambda:build-python3.7 image and package using make. The corresponding Makefile looks like this:

all: clean compile-libs debug

compile-libs:
        @docker run --rm \
          -v `pwd`:/src \
          -w /src \
          lambci/lambda:build-python3.7 \
          bash -c '''yum update -y\
            && yum groupinstall "Development Tools" -y\
            && yum install -y ibffi openssl-devel\
            && pip3 install virtualenv\
            && export VIRTUAL_ENV=/src/venv\
            && python3 -m venv $${VIRTUAL_ENV}\
            && export PATH="$${VIRTUAL_ENV}/bin:$${PATH}"\
            && pip3 install --upgrade pip\
            && pip3 install --upgrade --force -r ./requirements.txt -t .''' \
        && rm -rf venv

clean:
        @rm -rf enum*; find . -name '*.so' -delete

debug:
        @find . -name '*.so'

.PHONY: all

Once your subordinate private CA is up and running in ACTIVE state, you could issue_certificate to any server with access to openssl and curl, by generating a CSR and then chaining aws acm-pca issue-certificate ... followed by aws acm-pca certificate-issued ... and lastly aws acm-pca get-certificate ....

After the certificate is installed in your web server/application, all that is left to do is to make sure your clients trust your root CA, by importing its certificate into the corresponding certificate store on each client.

A demo stack is available as a starting point and can be extended with additional custom resources to issue certificates, etc.

--belodetek 😬