AWS Database Blog

Use Key Management Service (AWS KMS) to securely manage Ethereum accounts: Part 2

Ethereum is a popular public blockchain that makes it possible to create unstoppable applications in a permissionless fashion. It’s available to every user that has an Ethereum account. These Ethereum accounts consist of a private and an associated public key.

The main challenge as a user participating in a public blockchain such as Ethereum is safely managing the blockchain credentials. Because an externally owned Ethereum account is required to approve transactions, including funds transfers and other sensitive operations, its key material must be carefully safeguarded.

In some fully decentralized applications, the user is expected to manage their own key material. There are other applications, however, where it may desirable to entrust the management of key material to an external process or service, such as when the key material is frequently needed even when the user isn’t available. This is a common requirement for token staking and other modern blockchain applications.

This is the second of a two-post series about how to use AWS Key Management Service (AWS KMS) to manage Ethereum accounts.

The first post discussed how to do the following:

  • Use the AWS Cloud Development Kit (AWS CDK) to define AWS infrastructure
  • Apply custom configuration to AWS CDK objects
  • Use a Docker-based build for AWS Lambda functions and integrate it into the AWS CDK
  • Create signed Ethereum transactions based on the AWS customer master key (CMK)
  • Integrate the presented sample solution with an Amazon Managed Blockchain Ethereum Rinkeby testnet node.

This post discusses the following:

  • How Ethereum signatures work
  • How to use the AWS CMK to create an Ethereum public address
  • How to create Ethereum offline signatures using a CMK and make them portable

The third post in this series includes a deep dive into the Ethereum Improvement Proposal 1559 (EIP-1559) and explains how EIP-1559 transactions can be signed using AWS KMS.

Prerequisites

To follow along with this post, we recommend reading the first post and deploying the described AWS CDK-based solution.

To just make the source code available to your local system, you have to clone the repository from GitHub:

git clone https://github.com/aws-samples/aws-kms-ethereum-accounts.git

Lambda function (Ethereum key calculation)

The entire key handling logic is located in the eth-kms-client Lambda function. The source code to the Lambda function is located in the /aws-kms-ethereum-accounts/aws_kms_lambda_ethereum/_lambda/functions/eth_client folder.

Opening the lambda_function.py gives you an overview of how the requests are being handled. The lambda_handler(event, context) function expects a JSON request with an operation parameter defined. Based on this operation parameter, the handler runs the requested logic.

In this example, status and sign are supported:

operation = event.get('operation')
	if not operation:
        raise ValueError('operation needs to be specified in request and needs to be eigher "status" or "send"')

    if operation == 'status':
        
        [...]

        eth_checksum_address = ...
        
        return {'eth_checksum_address': eth_checksum_address}


    elif operation == 'sign':

        if not (event.get('amount') and event.get('dst_address') and event.get('nonce')):
            return {'operation': 'sign',
                    'error': 'missing parameter - sign requires amount, dst_address and nonce to be specified'}
        [...]
        
        raw_tx_signed = ...
        
        return {"signed_tx": raw_tx_signed}

If you examine the implementation of the sign operation more closely, the following steps are being run:

# get key_id from environment varaible
key_id = os.getenv('KMS_KEY_ID')

# get destination address from send request
dst_address = event.get('dst_address')

# get amount from send request
amount = event.get('amount')

# nonce from send request
nonce = event.get('nonce')

# download public key from KMS
pub_key = get_kms_public_key(key_id)

# calculate the Ethereum public address from public key
eth_checksum_addr = calc_eth_address(pub_key)

# collect raw parameters for Ethereum transaction
tx_params = get_tx_params(dst_eth_addr=dst_address,                          
                          amount=amount,
                          nonce=nonce)

# assemble Ethereum transaction and sign it offline
raw_tx = assemble_tx(tx_params=tx_params,
                     params=params,
                     eth_checksum_addr=eth_checksum_addr)

return {"signed_tx": raw_tx_signed}

Let’s go over these steps in more detail.

Because the AWS KMS-CMK instance is being created at the beginning, you can’t have a fixed reference to the key_id. To solve this problem, the key_id is passed to the Lambda function as a configuration parameter in form of an environment variable using KMS_KEY_ID as the variable name.

Lambda stores environment variables securely by encrypting them at rest. See the following code:

# get key_id from environment variable
key_id = os.getenv('KMS_KEY_ID')

The Lambda function itself is not tied to a single Ethereum account. It can be extended to support multiple accounts by securely providing key_ids to different AWS KMS-CMK instances within the AWS account. The destination AWS KMS-CMK instance would then need to be configured dynamically for each signing request. AWS Systems Manager Parameter Store should be used to securely store parameters and secrets. For more information see Sharing Secrets with AWS Lambda Using AWS Systems Manger Parameter Store.

The values for dst_address, amount, and nonce have to be passed using the external event triggering the Lambda function as described at the beginning:

# get destination address from send request
dst_address = event.get('dst_address')

# get amount from send request
amount = event.get('amount')

# nonce from send request
nonce = event.get('nonce')

The CMK public key is downloaded using the passed key_id. The implementation of the get_kms_public_key() method is located in the lambda_helper.py together with the lambda_function.py file. See the following code:

# download public key from KMS
pub_key = get_kms_public_key(key_id)

As you can see, the download step consists of the initialization of a new Python Boto3 client for the kms service. This client is then used to run the get_public_key() API call using the key_id that was passed. This step requires the kms:GetPublicKey permission on the CMK resource. See the following code:

def get_kms_public_key(key_id: str) -> bytes:
    client = boto3.client('kms')

    response = client.get_public_key(
        KeyId=key_id
    )

    return response['PublicKey']

Based on the public key that has been downloaded from the CMK instance, we can now calculate the checksum form of the public Ethereum address.

# calculate the Ethereum public address from public key
eth_checksum_addr = calc_eth_address(pub_key)

First, we need to decode the public key based on the underlying ASN.1 schema. This schema definition is assigned to the SUBJECT_ASN variable. We use the asn1tools package in this example to compile the schema and decode the key:

def calc_eth_address(pub_key) -> str:
    SUBJECT_ASN = '''
    Key DEFINITIONS ::= BEGIN

    SubjectPublicKeyInfo  ::=  SEQUENCE  {
       algorithm         AlgorithmIdentifier,
       subjectPublicKey  BIT STRING
     }

    AlgorithmIdentifier  ::=  SEQUENCE  {
        algorithm   OBJECT IDENTIFIER,
        parameters  ANY DEFINED BY algorithm OPTIONAL
      }

    END
    '''

    key = asn1tools.compile_string(SUBJECT_ASN)
    key_decoded = key.decode('SubjectPublicKeyInfo', pub_key)

    pub_key_raw = key_decoded['subjectPublicKey'][0]
    pub_key = pub_key_raw[1:len(pub_key_raw)]

    hex_address = w3.keccak(bytes(pub_key)).hex()
    eth_address = '0x{}'.format(hex_address[-40:])
   
    eth_checksum_addr = w3.toChecksumAddress(eth_address)

    return eth_checksum_addr

The schema definition can be found in IETF RFC5280If the decode step was successful, the raw public key can be accessed using a Python dictionary using subjectPublicKey as the key value.
It’s important to point out that according to IETF RFC5280, “the first octet of the OCTET STRING indicates whether the key is compressed or uncompressed. The uncompressed form is indicated by 0x04 and the compressed form is indicated by either 0x02 or0x03.”

Now the Keccak-256 hash value has to be taken from the raw public key. Furthermore, the Ethereum public address is defined to be the last 20 bytes (least signification bytes) of the hash value as stated in Chapter 4 of Mastering Ethereum.

Because the Python web3 library is being used in this example, we recommend using the provided Keccak method: w3.keccak().

After attaching the 0x prefix at the beginning of the hex string, we have successfully calculated our Ethereum public address.

The last step is to convert the address into a checksum address. Checksum addresses have been specified in EIP-55 and provide a basic protection against typos and copy/paste errors in Ethereum address handling.

The conversion is performed using the w3.toChecksumAddress() method.

We can now use this address to, for example, send funds to the CMK-based Ethereum account. These are then reflected in the associated state of the calculated addresses on the Ethereum blockchain.

get_tx_params() returns a Python dictionary containing the transaction-related parameters like the amount of ether to send, the destination address, and the nonce value, which acts as a replay protection:

# collect raw parameters for Ethereum transaction
tx_params = get_tx_params(dst_eth_addr=dst_address,
                          amount=amount,
                          nonce=nonce)
def get_tx_params(dst_eth_addr: str, amount: int, nonce: int) -> dict:
    transaction = {
        'nonce': nonce,
        'to': dst_eth_addr,
        'value': w3.toWei(amount, 'ether'),
        'data': '0x00',
        'gas': 160000,
        'gasPrice': '0x0918400000'
    }

    return transaction

assembe_tx() is called to assemble and to sign the transaction:

# assemble Ethereum transaction and sign it offline
raw_tx_signed = assemble_tx(tx_params=tx_params,
                            params=params,
                            eth_checksum_addr=eth_checksum_addr)

In detail, the assemble_tx() method (see the following code) consists of four high-level steps:

  • tx_usnsigned – An unsigned transaction is created
  • tx_sig – The hash value of the unsigned transaction is signed using AWS KMS and the resulting signature is decoded to extract and validate the values r and s
  • tx_eth_recovered_pub_addr – A missing signature parameter v is calculated
  • tx_encoded – A signed raw transaction is being assembled and returned in a serialized form
def assemble_tx(tx_params: dict, params: EthKmsParams, eth_checksum_addr: str) -> bytes:
    tx_unsigned = serializable_unsigned_transaction_from_dict(tx_params)
    tx_hash = tx_unsigned.hash()

    tx_sig = find_eth_signature(params=params,
                                plaintext=tx_hash)

    tx_eth_recovered_pub_addr = get_recovery_id(tx_hash, tx_sig['r'], tx_sig['s'], eth_checksum_addr)

    tx_encoded = encode_transaction(tx_unsigned,
                                    vrs=(tx_eth_recovered_pub_addr['v'], tx_sig['r'], tx_sig['s']))

    return tx_encoded

Let’s walk through these four steps in detail.

  1. An unsigned transaction is created using the web3 serializable_unsigned_transaction_from_dict()method:
tx_unsigned = serializable_unsigned_transaction_from_dict(transaction_dict=tx_params)
  1. The hash value of the unsigned transaction is signed using AWS KMS:
tx_sig = find_eth_signature(params=params,plaintext=tx_hash)

Looking into the find_eth_signature() method, you can see that the signature created by the CMK is returned in an ASN.1 schema. This schema needs to be decoded to get access to the values r and s. The schema definition for ECDSA signatures can be found in RFC3279 section 2.2.3.

SIGNATURE_ASN = '''
	Signature DEFINITIONS ::= BEGIN

    Ecdsa-Sig-Value  ::=  SEQUENCE  {
           r     INTEGER,
           s     INTEGER  }

    END
    '''
    signature_schema = asn1tools.compile_string(SIGNATURE_ASN)

    signature = sign_kms(params.get_ksm_key_id(), plaintext)

    # https://tools.ietf.org/html/rfc3279#section-2.2.3
    signature_decoded = signature_schema.decode('Ecdsa-Sig-Value', signature['Signature'])
    s = signature_decoded['s']
    r = signature_decoded['r']

With regards to the signing operation sign_kms(), it’s important to point out that the returned ECDSA signature is different every time it’s calculated, even though the same payload is being used. The reason for that is because AWS KMS doesn’t use Deterministic Digital Signature Generation (DDSG) and certain parameters in the signature calculation process are chosen random, namely the k-value.

The consequence of using random parameter k for the signature calculation is that the returned Ethereum signature is different every time, even using the same payload, as mentioned already.

After we extract r and s successfully, we have to test if the value of s is greater than secp256k1n/2 as specified in EIP-2 and flip it if required.

secp256_k1_n_half = SECP256_K1_N / 2

    if s > secp256_k1_n_half:
        s = SECP256_K1_N - s

The constant SECP256_K1_N represents the max value for s defined for the particular elliptic curve, as specified in Standards for Efficient Cryptography.

  1. The missing parameter v is being recovered using the Account.recoverHash() function of the eth_account Python package. v is also referred to as the recovery parameter.
tx_eth_recovered_pub_addr = get_recovery_id(msg_hash=tx_hash,
                r=tx_sig['r'], 
                s=tx_sig['s'], 
                eth_checksum_addr=eth_checksum_addr)

This parameter is important because Ethereum determines the sender’s public address based on the signature parameters r, s, and v. The function to determine the sender’s address from an assembled and signed transaction is important, so that an Ethereum peer can, for example, check if an account has sufficient funds for the gas costs associated with an Ethereum transaction.

Bitcoin, for example, uses the same cryptographic parameters but doesn’t require parameter v because it attaches the sender’s public address to the transaction. Ethereum avoids attaching the sender’s public address to save some bytes on the transaction size.

As stated in EIP-155, v is supposed to be determined based on the ChainID parameter to prevent replay attacks between different Ethereum networks like the Ethereum main net and the Rinkeby testnet.

The proposed way to determine v is: “v of the signature MUST be set to {0,1} + CHAIN_ID * 2 + 35, where {0,1} is the parity of the y value of the curve point for which r is the x-value in the secp256k1 signing process.”

Because this described approach relies on the CMK-based signature, we can’t calculate v as specified.

Instead, we can use a fallback mechanism specified in EIP-155. Here it’s stated that “the currently existing signature scheme using v = 27 and v = 28 remains valid and continues to operate under the same rules as it did previously.”

Because the public address, the hash value of the payload, and the values r and s are known, we can calculate the missing parameter v.

The eth_account Python package provides a function Account.recoverHash() that consumes a message hash and the parameters v, r, and s:

def get_recovery_id(msg_hash, r, s, expected_eth_addr) -> dict:
    for v in [27, 28]:
        recovered_addr = Account.recoverHash(message_hash=msg_hash,
                                             vrs=(v, r, s))

        if recovered_addr == expected_eth_addr:
            return {'recovered_addr': recovered_addr, 'v': v}

    return {}

As shown in the preceding code, we can run the recoverHash() twice with the values v=27 and v=28. If there is a collision with the passed Ethereum checksum address that was calculated earlier, the right value v has been determined.

If there is no match, something is wrong with either the previously calculated signatures or the payload.

  1. The last step consists of assembling a signed raw transaction, based on the unsigned_transaction value and the calculated signature values r, s, and v
  2. . To do so, we can use the encode_transaction() method provided by the eth_account Python package. This method returns a serialized version of the transaction object:
tx_encoded = encode_transaction(unsigned_transaction=tx_unsigned,
vrs=(tx_eth_recovered_pub_addr['v'], tx_sig['r'], tx_sig['s']))

The last step in the Lambda function consists of decoding the signed and serialized Ethereum transaction as a hexadecimal string and returning it embedded in a Python dictionary, which is returned as a JSON object by Lambda:

return w3.toHex(tx_encoded)

The hexadecimal string format is required to prevent encoding and escaping problems while moving the raw transaction payload around.

This signed transaction can now be used using the Ethereum JSON RPC eth_sendRawTransaction method, for example together with the Managed Blockchain for Ethereum nodes.

Keep in mind that the calculated AWS KMS CMK-based Ethereum address per default doesn’t have any funds associated with it, so it needs to be funded first.

Clean up

To avoid incurring future charges, delete the resources using the AWS CSK with the following command:

cdk destroy

You can also delete stacks deployed by the AWS CDK via the AWS CloudFormation console.

Conclusion

This series of posts discussed managing Ethereum key material using a CMK and Lambda.

In the first post, we talked about how to configure and deploy the required services using the AWS CDK. we explained how to configure an Ethereum-compatible CMK and how to extend the solution to send transactions to the Ethereum Rinkeby testnet using a Managed Blockchain Ethereum node.

In this second post, we explained the inner workings of the Ethereum signature process and showed how to use the created CMK resource to derive a public Ethereum address, how to create valid Ethereum offline signatures, and how to make these signatures portable.

About the Author

David Dornseifer is a Blockchain Architect with the Amazon ProServe Blockchain team. He focuses on helping customers design, deploy and scale end-to-end Blockchain solutions.