AWS Database Blog

Automating Hyperledger Fabric chaincode deployment on Amazon Managed Blockchain using AWS CodePipeline

Amazon Managed Blockchain is a fully managed service that makes it easy to create and manage blockchain networks using the open-source blockchain framework Hyperledger Fabric. In this post, we explore how to use AWS CodePipeline and related services to automate the deployment of chaincode.

Chaincode is a program that typically handles business logic agreed to by members of the network and is sometimes called a smart contract. To install and instantiate chaincode on the blockchain network, you use the open-source Hyperledger Fabric CLI or SDK with the endpoints exposed on your Hyperledger Fabric resources. Chaincode runs in an isolated Docker container and can interact with the ledger by submitting transactions to the fabric network. Unlike other blockchain platforms, chaincode isn’t written in a domain specific language (DSL). It’s written in Go, NodeJS, or Java (or languages running on those respective platforms, such as Typescript or Scala). This enables you to use standard test and build tools with chaincode as part of a CI/CD process. By doing so, we can make chaincode deployments automated and consistent.

In this post, we examine how to use tools such as CodePipeline, AWS CodeBuild, and AWS Lambda to perform automated testing and deployment of chaincode to Managed Blockchain.

Solution overview

The following architecture diagram describes our solution.


A developer first commits their chaincode program to AWS CodeCommit. This commit triggers a pipeline in CodePipeline.

Within the pipeline, CodeBuild is sent the chaincode that was committed to CodeCommit and performs a series of tests. If the tests pass, it zips up the code and stores it as a build artifact.

CodePipeline invokes a Lambda function that retrieves the build artifact, extracts it, and deploys it to Managed Blockchain. AWS Secrets Manager stores the credentials required to authenticate with Hyperledger Fabric and tracks the current version number of the chaincode.

The solution is hosted in the following GitHub repo.

Although our Lambda function is written in NodeJS, the Fabric libraries that it uses include native code. All native code libraries must be compiled in an Amazon Linux environment. The easiest way to do this is to use an AWS Cloud9 instance or an Amazon Elastic Compute Cloud (Amazon EC2) instance running Amazon Linux. In this environment, clone the code for this post:

git clone https://github.com/aws-samples/amazon-managed-blockchain-cicd-sample

The solution is provided as an AWS Cloud Development Kit (AWS CDK) application. The cdk folder contains the app. This app creates the architecture described earlier. It also creates AWS Identity and Access Management (IAM) resources for this architecture. For more information about the AWS CDK, see Getting started with the AWS CDK.

Prerequisites

This walkthrough assumes you have already set up the following:

When you have the prerequisites installed, you can proceed to the next step.

Examining the Lambda function

The Lambda function reads the location of the chaincode build artifact from the event passed from CodePipeline, obtains Fabric credentials from Secrets Manager, and installs the chaincode to Fabric. It is written in NodeJS and is in the lambda directory.

Downloading dependencies

Prior to deploying our solution, we have to download the dependencies for our Lambda function. Change to the lambda directory within the solution and then enter the following code:

npm install

Uploading certificates

The Lambda function needs an admin identity in order to install chaincode to the Fabric network. To provide this, we upload Fabric credentials to Secrets Manager using the AWS Command Line Interface (AWS CLI). The Lambda function accesses the secrets and uses them to authenticate with Fabric and deploy chaincode. To upload the credentials to Secrets Manager, run the following commands on your Fabric client node. Provide the name of your Fabric member as $MEMBERNAME, and substitute the path for the correct location of your admin-msp:

aws secretsmanager create-secret --name "dev/fabricOrgs/$MEMBERNAME/admin/pk" - secret-string "`cat /home/ec2-user/admin-msp/keystore/*`" --region us-east-1

aws secretsmanager create-secret --name "dev/fabricOrgs/$MEMBERNAME/admin/signcert" --secret-string "`cat /home/ec2-user/admin-msp/signcerts/*`" --region us-east-1

Providing a connection profile

A connection profile is a YAML file that provides the URL endpoints for the peer, ordering service, CA, and other details required for your Lambda function to connect to your Fabric network. The gen-connection-profile.sh script can generate a connection profile from shell variables.

Alternatively, you can manually generate a connection profile. For instructions, see fabric-client: How to use a common connection profile. You can obtain the information necessary to populate the connection profile from the Managed Blockchain console. Name your connection profile connection-profile.yaml and place it in the lambda directory.

Downloading Managed Blockchain certs

We need to obtain the latest version of the Managed Blockchain PEM file and place it in the certs directory within the Lambda function source. To do so, run the following command in the lambda folder. Replace MyRegion with the AWS Region you’re using—for example, us-east-1:

aws s3 cp s3:// MyRegion.managedblockchain/etc/managedblockchain-tls-chain.pem ~/certs

Deploying the AWS CDK app

You need to change several variables in the cdk.ts file prior to deploying the app:

  • CA endpoint
  • Peer endpoint
  • Order endpoint
  • MSP ID
  • Member name
  • VPC ID

You can find the details pertinent to Managed Blockchain from the client node or from your blockchain network details on the Managed Blockchain console. The VPC ID is the ID for the VPC (for example, vpc-12345) that you created earlier. Modify the variables to reflect your environment.

When the variables are up to date, in the cdk folder, run the following:

cdk deploy

You receive the following prompt:

Do you wish to deploy these changes (y/n)?

Accept the changes and the solution is deployed via AWS CloudFormation. After the solution is deployed, navigate to the CodePipeline console and you can see that a new pipeline has been created. This pipeline has three stages: source, build, and deploy, and correlates to the architecture diagram from earlier. We return to the first stage soon to start the pipeline, but first let’s examine the build and deploy stages.

Testing chaincode with CodeBuild

The chaincode for this solution resides in the chaincode folder. It’s a very simple golang program named forex, which you can use to get and set foreign exchange rates, such as (“USD:CAD”, 0.75). We test this chaincode and then deploy it to Fabric.

CodeBuild uses a YAML buildspec file to run a build for this chaincode. The buildspec file for our solution is in the chaincode folder, which is included inline in the following code:

version: 0.2

env:
  variables:
    PACKAGE: "forex"
phases:
  install:
    runtime-versions:
      golang: 1.13
    commands:
      # Download the appropriate version of the Fabric Shim
      - wget https://github.com/hyperledger/fabric/archive/v1.4.2.tar.gz
      - tar xvzf v1.4.2.tar.gz
      - mkdir -p /go/src/github.com/hyperledger
      - mv fabric-1.4.2 /go/src/github.com/hyperledger/fabric
      - rm v1.4.2.tar.gz
      
      # Install golint
      - go get -u github.com/golang/lint/golint
  build:
    commands:
      - mkdir -p /go/src/${PACKAGE}
      - ls /go/src/forex
      - ls ${CODEBUILD_SRC_DIR}

      - cp -r ${CODEBUILD_SRC_DIR}/* /go/src/${PACKAGE}/

      - cd /go/src/${PACKAGE}

      - go get -t .
      
      # Lint our Chaincode 
      - golint -set_exit_status

      # Check the Go code for common problems with 'go vet'
      - go vet .

      # Run unit tests using Mockshim to test our chaicode
      - go test .

      - zip -r  /go/src/${PACKAGE}/forex.zip "/go/src/${PACKAGE}"

artifacts:
  files:
     - /go/src/${PACKAGE}/forex.zip

This differs from a traditional golang build in a significant way: the end artifact isn’t a binary. Rather, it’s a .zip file of our chaincode. The reason for this is that the chaincode is compiled as part of the installation process. Instead, this buildspec file runs tests against the chaincode and produces a .zip file that you can use to install the chaincode on your Fabric network.

The buildspec defines the following steps:

  • GoLint – Detects style mistakes
  • GoVet – Detects common go problems such as inappropriate calls to printf or improper pointer passing
  • Go test – Runs our test suite based around shimtest

The first two steps are agnostic to the chaincode; they simply treat the code as standard go code. Our test suite, however, uses shimtest to perform unit tests against our chaincode.

shimtest provides a mock version of the shim.ChainCodeStubInterface and can simulate much of the interactions between our chaincode and Hyperledger Fabric. We can use shimtest to ensure that our chaincode implements the correct business logic and that the appropriate state changes are written to the ledger. This allows us to validate the correctness of our chaincode without having to deploy the chaincode to Fabric. This approach is fully compatible with traditional unit testing, and allows you to use standard tools to run individual tests. Because our chaincode is written in golang, we can make use of the golang testing and assert packages to orchestrate our tests using shimtest. This allows us to test the chaincode with the following code:

go test

Instead of running the test manually, we use CodeBuild. Before we proceed, let’s examine the simple test that we run on our chaincode. In the chaincode folder, the forex_test.go file contains a simple function for testing our chaincode using MockStub. The following line creates a mock instantiation of our chaincode, which allows us to perform mock invocations:

stub := shim.NewMockStub("forex", new(ForexChaincode))

The following lines run the invocation and use the assert library to confirm that the result is accurate:

response = stub.MockInvoke(uuid, args)
assert.EqualValues(t, shim.OK, response.GetStatus(), "failed to execute invocation")

Committing chaincode and starting the pipeline

To use the pipeline, we have to commit the contents of the chaincode directory to the CodeCommit repository. You can find the name and URL of the CodeCommit repo from the AWS CDK stack outputs or the CodeCommit console.

You can commit to the repo with either a Git client or the AWS CLI. For instructions, see Create a commit in AWS CodeCommit.

Committing to the CodeCommit repo triggers the pipeline. The chaincode should successfully pass testing, after which the Lambda function installs the chaincode to Managed Blockchain. The CodePipeline console should show a result similar to the following screenshot.

If you inspect the logs for CodeBuild, you should see a successful run where the chaincode passed linting, go vet, and testing. Likewise, the Lambda logs should show that it has successfully installed the chaincode to the Fabric network.

You can confirm that the installation completed successfully by listing the installed chaincodes from a Fabric client node:

peer chaincode list --installed

You should see something resembling the following code:

Name: forex, Version: 1, Path: forex, Id …

Subsequently, you can instantiate the chaincode by running the following command on your fabric client:

docker exec -e "CORE_PEER_TLS_ENABLED=true" \
-e "CORE_PEER_TLS_ROOTCERT_FILE=/opt/home/managedblockchain-tls-chain.pem"  \
-e "CORE_PEER_LOCALMSPID=$MSP" \
-e "CORE_PEER_MSPCONFIGPATH=$MSP_PATH"  \
-e "CORE_PEER_ADDRESS=$PEER" \
cli peer chaincode install -n $FOREXCHAINCODENAME -v v0 -p forex/cmd

Clean Up

When you have completed the solution, you can delete the AWS CDK stack with the following code:

cdk destroy

If you created a new VPC, AWS Cloud9 instance, or other resources, you can also delete those.

Summary

In this post, we explored how to use CodePipline and Managed Blockchain to automated the testing and deployment of chaincode to a Hyperledger Fabric network.

You could extend this solution further through additional Lambda functions to instantiate chaincode, create channels, perform integration testing, and other activities as part of the pipeline.

About the Author

Dr. Jonathan Shapiro-Ward is a Principal Solutions Architect at AWS based in Toronto. Jonathan has been with AWS for 3.5 years and in that time has worked helping customers solve problems including petabyte scale analytics and the adoption of ML, serverless, and blockchain. He has spoken at events across the world where he focused on areas of emerging technology. Jonathan has previously worked in a number of industries including gaming and fintech and has a background in academic research. He holds a PhD in distributed systems from the University of St Andrews.