AWS DevOps & Developer Productivity Blog

AWS CodeBuild Managed Self-Hosted GitHub Action Runners

AWS CodeBuild now supports managed self-hosted GitHub Action runners, allowing you to build powerful CI/CD capabilities right beside your code and quickly implement a build, test and deploy pipeline. Last year AWS announced that customers can define their GitHub Actions steps within any phase of a CodeBuild buildspec file but with a self-hosted runner, jobs execute from GitHub Actions on GitHub.com to a system you deploy and manage.

With the recent announcement that AWS CodeBuild now supports managed GitHub Action runners, AWS can take care of managing the hosting of GitHub Action self-hosted runners within CodeBuild allowing teams to run their GitHub Actions workflow jobs natively within AWS.

For customers managing their self-hosted runners on their own infrastructure, CodeBuild can now provide a secure, scalable and lower latency solution. In addition, CodeBuild managed self-hosted GitHub Action runners bring features, such as:

With the compute options available, customers can now run tests on hardware and operating system combinations that closely match production and reduce manual operational tasks by shifting the management of the runners to AWS.

In this blog, I will explore how AWS managed GitHub Action self-hosted runners work by building and deploying an application to AWS using GitHub Actions.

Architecture overview

The architecture of what I’ll be building can be seen below:

Architecture diagram of AWS CodeBuild running managed GitHub Actions Runners

Figure 1. Architecture diagram of AWS CodeBuild running managed Self-Hosted GitHub Actions Runners

The architecture above shows how a developer pushes code changes to GitHub. This triggers CodeBuild to detect the update. CodeBuild then runs the defined GitHub Action Workflow, which builds and deploys it to AWS Lambda.

Step 1. Build a AWS Lambda Function

I’ll start with a simple application to demonstrate how to build and deploy an application on AWS via a Managed Self-Hosted GitHub Actions runner. We’ve written before about why AWS is the best place to run Rust, Amazon CTO Werner Vogels has been an outspoken advocate for exploring energy-efficient programming languages like Rust and AWS have great guides on using Rust to build on AWS such as:

Cargo Lambda is one of the simplest ways to run, build and deploy Rust lambda functions on AWS, I’ll start with the Getting Started guide:

  1. Navigate to GitHub.com and create a new GitHub repository

    Create a new GitHub repository

    Figure 2 Create a new GitHub repository

  2. Clone the repository locally: 
    git clone https://github.com/{{user-name}}/rust-api-demo.git
  3. From the above cloned repository, install Cargo Lambda:For macOS & Linux: 
    brew tap cargo-lambda/cargo-lambda
brew install cargo-lambda

    Windows users can follow the guide to see all the ways that you can install Cargo Lambda in your system.

  4. Use Cargo lambda to create a new project 
    cargo lambda new new-lambda-project && cd new-lambda-project

It’s now possible to explore the project, in this case I am using JetBrains RustRover with Amazon Q Developer installed to increase my productivity while working on the application:

JetBrains RustRover with Amazon Q Developer

Figure 3. JetBrains RustRover with Amazon Q Developer

Amazon Q Developer is available on a free tier and provides real-time code suggestions as well as advanced suggestions such as in-built chat to reason with the code we’re working on.

  1. Add, Commit & Push the code to your GitHub account: 
    git add .
git commit -m “Initial Commit”
git push origin

Step 2. Create AWS CodeBuild Project

The AWS Documentation outlines how to set up self-hosted GitHub Actions runners in AWS CodeBuild, the key here is to setup a GitHub webhook event of event type WORKFLOW_JOB_QUEUED so that CodeBuild will only process GitHub Actions workflow jobs.

I will create a new CodeBuild project as per the documentation to connect CodeBuild to our GitHub repository and correctly configure a webhook to trigger the GitHub Actions.

  1. Open the AWS CodeBuild console
  2. Create a build project.
    • In Source:
      • For Source provider, choose GitHub.
      • For Repository, choose Repository in my GitHub account.
      • For Repository URL, enter https://github.com/user-name/repository-name.
    • In Primary source webhook events:
      • For Webhook – optional, select Rebuild every time a code change is pushed to this repository.
      • For Event type, select WORKFLOW_JOB_QUEUED. Once this is enabled, builds will only be triggered by GitHub Actions workflow jobs events.

        WORKFLOW_JOB_QUEUED Event Type

        Figure 4. WORKFLOW_JOB_QUEUED Event Type

    • In Environment:
      • Choose a supported Environment image and Compute. Note that you have the option to override the image and instance settings by using a label in your GitHub Actions workflow YAML.
    • In Buildspec:
      • Note that your Buildspec will be ignored. Instead, CodeBuild will override it to use commands that will setup the self-hosted runner. This project’s primary responsibility is to set up a self-hosted runner in CodeBuild to run GitHub Actions workflow jobs.
  3. Continue with the remaining default options and select Create build project.

CodeBuild Service Role Permissions

In order for the CodeBuild service role to be able to successfully create and deploy a Lambda function, the service role will require the necessary permissions. The Service role can be seen when editing the CodeBuild project:

CodeBuild Service Role Permissions

Figure 5. CodeBuild Service Role Permissions

The required Lambda permissions are documented in the Cargo Lambda documentation:

  • lambda:GetFunction
  • lambda:CreateFunction
  • lambda:UpdateFunctionCode

In addition, there are also IAM permissions required:

  • iam:CreateRole
  • iam:AttachRolePolicy
  • iam:UpdateAssumeRolePolicy
  • iam:PassRole

Add the required permissions to the service role for the CodeBuild project:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:AttachRolePolicy",
                "iam:UpdateAssumeRolePolicy",
                "iam:PassRole"
            ],
            "Resource": [
                "arn:aws:iam::{AWS:Account}:role/AWSLambdaBasicExecutionRole",
                "arn:aws:iam::{AWS:Account}:role/cargo-lambda-role*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:UpdateFunctionCode",
                "lambda:GetFunction"
            ],
            "Resource": "arn:aws:lambda::{AWS:Account}:function:{function-name}"
        }
    ]
}

Note that I do not need to manage IAM permissions outside of our AWS Account, for example GitHub does not need to know about our AWS permissions.

Step 3. Create a GitHub Action Workflow

GitHub Actions is a continuous integration and continuous deliver (CI/CD) platform that provides automation through building, testing and deploying applications. In this section we will create a GitHub Action Workflow to build and deploy our Lambda.

  1. Navigate back to our GitHub project create a workflow within the .github/workflows directory, the Simple Workflow is a good starting point:

    Create a Simple Workflow

    Figure 6. Create a Simple Workflow

  2. Update the Job to include the tooling required to build our Rust Lambda function, the details can be found in the GitHub Actions section. Our workflow file should now look like this:
name: rust-api-demo-cicd

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

env:
  CARGO_TERM_COLOR: always

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Install Rust toolchain
        uses: dtolnay/rust-toolchain@stable
      - name: Install Zig toolchain
        uses: korandoru/setup-zig@v1
        with:
          zig-version: 0.10.0
      - name: Install Cargo Lambda
        uses: jaxxstorm/action-install-gh-release@v1.12.0
        with:
          repo: cargo-lambda/cargo-lambda
          tag: v0.14.0 
          platform: linux 
          arch: x86_64 
          # Add your build steps below

The above GitHub Actions Workflow currently runs on GitHub; However, I now want to make two further changes:

  • Define an AWS CodeBuild runner
  • Define Build and Deploy Lambda steps

Define an AWS CodeBuild runner

A GitHub Actions workflow is made up of one or more jobs, each job runs in a runner environment specified by runs-on. The value for runs-on to specify CodeBuild as a runner takes the format:

runs-on: codebuild-<CodeBuildProjectName>-${{ github.run_id }}-${{ github.run_attempt }}

I will update the <CodeBuildProjectName> to the CodeBuild project name that was entered in Step2, e.g. “GitHubActionsDemo”.

When configuring CodeBuild as a runner environment, BuildSpecs are ignored. In order to define the specification of our build environments it is possible to pass in variables including:

  • EC2 compute builds: Image, image version, instance size
  • Lambda compute builds: environment type, runtime version, instance size

For further details, see the action runner guide.

Define Build and Deploy Lambda steps

The last change is to add steps to check out our code onto the runner and then build and deploy using cargo lambda:

      - name: Build Rust API
        uses: actions/checkout@v4
      - run: cargo lambda build --release
      - run: cargo lambda deploy

The final workflow looks like this:

name: rust-api-demo-cicd

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

env:
  CARGO_TERM_COLOR: always

jobs:
  build:
    runs-on: codebuild-GitHubActionsDemo-${{ github.run_id }}-${{ github.run_attempt }}
    steps:
      - name: Install Rust toolchain
        uses: dtolnay/rust-toolchain@stable
      - name: Install Zig toolchain
        uses: korandoru/setup-zig@v1
        with:
          zig-version: 0.10.0
      - name: Install Cargo Lambda
        uses: jaxxstorm/action-install-gh-release@v1.12.0
        with:
          repo: cargo-lambda/cargo-lambda
          tag: v0.14.0 
          platform: linux 
          arch: x86_64 
          # Add your build steps below
      - name: Build Rust API
        uses: actions/checkout@v4
      - run: cargo lambda build --release
      - run: cargo lambda deploy

When I commit changes to the workflow to the main branch it will trigger the GitHub Action.

Step 4. Testing our GitHub Action Workflow.

The GitHub Action is currently triggered on all push and pull requests to main branch:

Trigger a build

Figure 7. Trigger a build

Note that GitHub is where the CI/CD process is being driven, the build logs are available in GitHub as the job is running:

GitHub Action Logs

Figure 8. GitHub Action Logs

As the build progresses through the deployment step, the details of the Lambda function deployed are shown:

Deployment ARN

Figure 9. Deployment ARN

Navigating back to the AWS Console, the deployed Lambda Function can be seen:

Lambda Deployed

Figure 10. Lambda Deployed

And finally, opening the CodeBuild console, it’s possible to observe the status of the Managed GitHub Actions Runner, the build number and also the duration:

Lambda Deployed via Managed Self-Hosted GitHub Action runner

Figure 11. Lambda Deployed via Managed Self-Hosted GitHub Action runner

Clean Up

To avoid incurring future charges:

  1. Delete the Lambda created via the deployment in Step 4.
  2. Delete the CodeBuild Project created in Step 2.

Conclusion

As I’ve shown in this blog, setting up GitHub Actions Workflows that run on AWS is now even easier to allow CodeBuild projects to receive GitHub Actions workflow job events and run them on CodeBuild ephemeral hosts. AWS customers can take advantage of natively integrating with AWS and providing security and convenience through features such as defining service role permissions with AWS IAM or passing credentials as environment variables to build jobs with AWS Secrets Manager.

Being able to use CodeBuild’s reserved capacity allows you to provision a fleet of CodeBuild hosts that persist your build environment. These hosts remain available to receive subsequent build requests, which reduces build start-up latencies but also make it possible to compile your software within your VPC and access resources such as Amazon Relational Database Service, Amazon ElastiCache, or any service endpoints that are only reachable from within a specific VPC.

CodeBuild-hosted GitHub Actions runners are supported in all CodeBuild regions and customers managing their CI/CD processes via GitHub Actions can use the compute platforms CodeBuild offers, including Lambda, Windows, Linux, Linux GPU-enhanced and Arm-based instances powered by AWS Graviton Processors.

Read more in our documentation for GitHub Action runner in AWS CodeBuild.

About the author:

Matt Laver

Matt Laver is a Senior Solutions Architect at AWS working with SMB customers in EMEA. He is passionate about DevOps and loves helping customers find simple solutions to difficult problems.