AWS Partner Network (APN) Blog

An Introduction to AWS CodeBuild

This is a guest post by Harlen Bains, Stelligent. Stelligent is an AWS DevOps Competency Partner.

At re:Invent 2016, AWS announced a new service called AWS CodeBuild that allows you to build your software. This AWS managed build service can be used to compile your source code, run unit tests, and produce deployable application artifacts. In short, CodeBuild eliminates the need to manage, provision, scale, and maintain build servers. With support for most build languages and tools like Maven and Gradle, CodeBuild is ready to go out of the box. It provides built-in support for programming platforms such as Java, Ruby, Python, Golang, Docker, Node, and Android and can be customized for others. As with most AWS services you pay for only what you use and the service scales automatically based on your demand (for more information, see AWS CodeBuild pricing).

In this post we are going to explore the basics of CodeBuild and then learn how to use the service to build a Java application. In Figure 1, you see how CodeBuild fits into the build stage of a typical deployment pipeline. The pipeline itself can be orchestrated with a tool like AWS CodePipeline – which will be discussed in our next post.

Figure 1 – How CodeBuild fits into a deployment pipeline

Before we start building our own artifacts we need to understand two key CodeBuild concepts: Build Environment and the Build Spec File.

Build Environment – The operating system, programming language runtime, and tools that CodeBuild uses to run a build are specified by a Docker image. A customer can either use their own Docker image or use one of the many optimized images that are provided by AWS.

Build Spec File – A required YAML file that contains build commands and related settings that CodeBuild uses for a build.

In addition, CodeBuild relies on the following services and solutions:

Amazon S3, GitHub, and CodeCommit – Used to store source code, build spec, and build environment

IAM – Securely provides access to AWS resources

CloudWatch Logs – Used to store and access logs for each CodeBuild run

Although CodeBuild is usually used as the build component of AWS CodePipeline, it can also be used to replace the building component of Jenkins and other tools. Cross-tool support is provided through the AWS SDKs for CodeBuild.

By default, CodeBuild allows up to 60 minutes for a build, but this can be set to any amount of time between 5 minutes and 8 hours. Keep in mind that you will also be charged for any other AWS resources that are provisioned as part of CodeBuild. This might include services such as S3, KMS, CloudWatch, and other EC2 resources provisioned by CodeBuild. In figure 2, you see how CodeBuild can be executed via the AWS Management Console, AWS CLI, AWS SDKs, and AWS CodePipeline.

Figure 2 – Different ways to access CodeBuild

Who Should Use CodeBuild?

While CodeBuild can be seamlessly integrated with AWS CodePipeline, you can also integrate it with many other tools in your tooling ecosystem. The following scenarios are ideal candidates for using CodeBuild:

  • Are you still building JARs in your IDE?
  • Are you looking to remove the need to setup a separate Jenkins environment to build artifacts?
  • Do you want to reduce the amount of dedicated build infrastructure you maintain?
  • Are you trying to move to a hosted Continuous Integration (CI) server but it’s difficult to get new services approved within your organization?
  • Is your organization uncomfortable having a third party run their builds, but comfortable with running those builds in AWS?

Let’s Get CodeBuild-ing! (Tutorial)

Let’s take a look at what we want to do here. Starting with an AWS provided sample application source code, we are going to use CodeBuild to create a deployable Java artifact. In Figure 3 we can see how this looks as a process.

Figure 3 – CodeBuild process

Step 1 – Setup

Download this GitHub package: https://github.com/stelligent/aws-codedeploy-sample-tomcat (as shown in Figure 3) and upload the zip to an S3 Bucket. Be sure that S3 versioning is enabled for the bucket.

Take note of the S3 bucket name and the name of the zip file. You will be needing these later.

Step 2 – Create Project

Launch CodeBuild by going to https://console.thinkwithwp.com/codebuild/ and click Create project (or Get Started and then Create project if this is your first time using CodeBuild). Once you’ve clicked this, you will be see a page where you can enter the settings for your build project.

Step 3 – Configure Project

1.Let’s begin to configure the project. There are a few steps to this:

a. Project Configuration – Add a project name. Pick one that makes sense. This is what you will see on your CodeBuild page and in your logs.

b. Source: What to Build – Add the source for your code. Here we select S3 as a source and then enter the name of the S3 bucket that we created earlier in Step 1. You’ll enter aws-codedeploy-sample-tomcat-master.zip as the S3 object key. Although we are using S3 in this example we could also use GitHub or CodeCommit as well.

c. Environment: How to Build – Right after the source we want to tell CodeBuild how to build the file. The first step is to select Use an image managed by AWS CodeBuild for the Environment image*. This way we do not need to create our own Docker image. Next select the Operating system*. Here we use Ubuntu and then we select Java as our run time – since the source we are trying to build is a Java project.

In the version selection, we tell CodeBuild what version of Java we want to use. We are using aws/codebuild/java:openjdk-8.

For the Build specification section, select Insert build commands. Under the Build command* section, type:

mvn package

We use this build command because CodeBuild actually extracts the Zip file from S3 and uses the specific folder name. Once in that folder, we want to build from the Maven package in the source code.

buildspec.yml

While the above example demonstrates the use of running a simple build command, you can also provide a more expressive description of a build using a buildspec.yml file. In this YAML file you can configure the commands that occur in each of the build phases along with the name and type of the artifact file(s). The buildspec.yml file needs to reside in the root directory of the source repository. A snippet of the buildspec.yml is shown below:

version: 0.1
phases:
  build:
    commands:
      - mvn package
artifacts:
  files:
    - appspec.yml
    - target/SampleMavenTomcatApp.war
    - scripts/*

 

d. Artifacts: Where to Put the Artifacts from this Build Project – Now that we have told CodeBuild where to find the source code and how to build the artifact, we are going to tell it where to store the built artifact (i.e. a WAR file, in this case).

Under the Output Files section, enter:

appspec.yml, target/SampleMavenTomcatApp.war, scripts/*

We are going to store the artifact in S3, with the name CodeBuildTomcat, and in the same bucket we used to store the source code.

e. Service Role – Create a new Service Role in IAM for this CodeBuild project.

f. Click Continue

2. View the review page and then click Save and Build.

3. Now on the Start new build page just click Start build.

4.  On the next page wait for the build to succeed.

5. Once it has completed, you can open the S3 bucket you specified for the artifact and view the S3 bucket with the SampleMavenTomcatApp.war file.

CodeBuild Phases

As CodeBuild is building artifacts, it goes through several distinct phases that are logged in AWS CloudWatch Logs (and accessible from the CodeBuild console). Each of these phases are described in some more detail below:

    • SUBMITTED – This is the initial phase that indicates that the build process initiated and configuration has been received
    • PROVISIONING – In this phase, CodeBuild launches a build container using a Docker image as defined
    • DOWNLOAD_SOURCE – Downloads the source from S3, CodeCommit or GitHub
    • INSTALL – Installs the source onto the container
    • PRE_BUILD – Any actions that need to occur prior to the build
    • BUILD – Executes the commands defined in the build specification
    • POST_BUILD – Runs any cleanup actions
    • UPLOAD_ARTIFACTS – Uploads the build artifacts to S3
    • FINALIZING – Completing the build process
    • COMPLETED – The build process is complete

6. Congratulations! Now you have used CodeBuild to build a sample application.

Conclusion

Now that you have successfully created an artifact with CodeBuild, the next step is to run it with the rest of the AWS Developer Tools suite. To see how to do this, stay tuned for our next post: Deploy to Production using CodeBuild and Developer Tools Suite which demonstrates how to integrate CodeBuild with AWS CodeDeploy, AWS CodeCommit, and AWS CodePipeline by automating the provisioning in AWS CloudFormation.