AWS DevOps & Developer Productivity Blog
Create Multiple Builds from the Same Source Using Different AWS CodeBuild Build Specification Files
In June 2017, AWS CodeBuild announced you can now specify an alternate build specification file name or location in an AWS CodeBuild project.
In this post, I’ll show you how to use different build specification files in the same repository to create different builds. You’ll find the source code for this post in our GitHub repo.
Requirements
The AWS CLI must be installed and configured.
Solution Overview
I have created a C program (cbsamplelib.c) that will be used to create a shared library and another utility program (cbsampleutil.c) to use that library. I’ll use a Makefile to compile these files.
I need to put this sample application in RPM and DEB packages so end users can easily deploy them. I have created a build specification file for RPM. It will use make to compile this code and the RPM specification file (cbsample.rpmspec) configured in the build specification to create the RPM package. Similarly, I have created a build specification file for DEB. It will create the DEB package based on the control specification file (cbsample.control) configured in this build specification.
RPM Build Project:
The following build specification file (buildspec-rpm.yml) uses build specification version 0.2. As described in the documentation, this version has different syntax for environment variables. This build specification includes multiple phases:
- As part of the install phase, the required packages is installed using yum.
- During the pre_build phase, the required directories are created and the required files, including the RPM build specification file, are copied to the appropriate location.
- During the build phase, the code is compiled, and then the RPM package is created based on the RPM specification.
As defined in the artifact section, the RPM file will be uploaded as a build artifact.
version: 0.2
env:
variables:
build_version: "0.1"
phases:
install:
commands:
- yum install rpm-build make gcc glibc -y
pre_build:
commands:
- curr_working_dir=`pwd`
- mkdir -p ./{RPMS,SRPMS,BUILD,SOURCES,SPECS,tmp}
- filename="cbsample-$build_version"
- echo $filename
- mkdir -p $filename
- cp ./*.c ./*.h Makefile $filename
- tar -zcvf /root/$filename.tar.gz $filename
- cp /root/$filename.tar.gz ./SOURCES/
- cp cbsample.rpmspec ./SPECS/
build:
commands:
- echo "Triggering RPM build"
- rpmbuild --define "_topdir `pwd`" -ba SPECS/cbsample.rpmspec
- cd $curr_working_dir
artifacts:
files:
- RPMS/x86_64/cbsample*.rpm
discard-paths: yes
Using cb-centos-project.json as a reference, create the input JSON file for the CLI command. This project uses an AWS CodeCommit repository named codebuild-multispec and a file named buildspec-rpm.yml as the build specification file. To create the RPM package, we need to specify a custom image name. I’m using the latest CentOS 7 image available in the Docker Hub. I’m using a role named CodeBuildServiceRole. It contains permissions similar to those defined in CodeBuildServiceRole.json. (You need to change the resource fields in the policy, as appropriate.)
{
"name": "rpm-build-project",
"description": "Project which will build RPM from the source.",
"source": {
"type": "CODECOMMIT",
"location": "https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/codebuild-multispec",
"buildspec": "buildspec-rpm.yml"
},
"artifacts": {
"type": "S3",
"location": "codebuild-demo-artifact-repository"
},
"environment": {
"type": "LINUX_CONTAINER",
"image": "centos:7",
"computeType": "BUILD_GENERAL1_SMALL"
},
"serviceRole": "arn:aws:iam::012345678912:role/service-role/CodeBuildServiceRole",
"timeoutInMinutes": 15,
"encryptionKey": "arn:aws:kms:eu-west-1:012345678912:alias/aws/s3",
"tags": [
{
"key": "Name",
"value": "RPM Demo Build"
}
]
}
After the cli-input-json file is ready, execute the following command to create the build project.
version: 0.2
env:
variables:
build_version: "0.1"
phases:
install:
commands:
- apt-get install gcc make -y
pre_build:
commands:
- mkdir -p ./cbsample-$build_version/DEBIAN
- mkdir -p ./cbsample-$build_version/usr/lib
- mkdir -p ./cbsample-$build_version/usr/include
- mkdir -p ./cbsample-$build_version/usr/bin
- cp -f cbsample.control ./cbsample-$build_version/DEBIAN/control
build:
commands:
- echo "Building the application"
- make
- cp libcbsamplelib.so ./cbsample-$build_version/usr/lib
- cp cbsamplelib.h ./cbsample-$build_version/usr/include
- cp cbsampleutil ./cbsample-$build_version/usr/bin
- chmod +x ./cbsample-$build_version/usr/bin/cbsampleutil
- dpkg-deb --build ./cbsample-$build_version
artifacts:
files:
- cbsample-*.debHere we use cb-ubuntu-project.json as a reference to create the CLI input JSON file. This project uses the same AWS CodeCommit repository (codebuild-multispec) but a different buildspec file in the same repository (buildspec-deb.yml). We use the default CodeBuild image to create the DEB package. We use the same IAM role (CodeBuildServiceRole).
{
"name": "deb-build-project",
"description": "Project which will build DEB from the source.",
"source": {
"type": "CODECOMMIT",
"location": "https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/codebuild-multispec",
"buildspec": "buildspec-deb.yml"
},
"artifacts": {
"type": "S3",
"location": "codebuild-demo-artifact-repository"
},
"environment": {
"type": "LINUX_CONTAINER",
"image": "aws/codebuild/ubuntu-base:14.04",
"computeType": "BUILD_GENERAL1_SMALL"
},
"serviceRole": "arn:aws:iam::012345678912:role/service-role/CodeBuildServiceRole",
"timeoutInMinutes": 15,
"encryptionKey": "arn:aws:kms:eu-west-1:012345678912:alias/aws/s3",
"tags": [
{
"key": "Name",
"value": "Debian Demo Build"
}
]
}Using the CLI input JSON file, create the project, start the build, and check the status of the project.
After successful completion of the RPM and DEB builds, check the S3 bucket configured in the artifacts section for the build packages. Build projects will create a directory in the name of the build project and copy the artifacts inside it.
Override Buildspec During Build Start:
It’s also possible to override the build specification file of an existing project when starting a build. If we want to create the libs RPM package instead of the whole RPM, we will use the build specification file named buildspec-libs-rpm.yml. This build specification file is similar to the earlier RPM build. The only difference is that it uses a different RPM specification file to create libs RPM.
version: 0.2
env:
variables:
build_version: "0.1"
phases:
install:
commands:
- yum install rpm-build make gcc glibc -y
pre_build:
commands:
- curr_working_dir=`pwd`
- mkdir -p ./{RPMS,SRPMS,BUILD,SOURCES,SPECS,tmp}
- filename="cbsample-libs-$build_version"
- echo $filename
- mkdir -p $filename
- cp ./*.c ./*.h Makefile $filename
- tar -zcvf /root/$filename.tar.gz $filename
- cp /root/$filename.tar.gz ./SOURCES/
- cp cbsample-libs.rpmspec ./SPECS/
build:
commands:
- echo "Triggering RPM build"
- rpmbuild --define "_topdir `pwd`" -ba SPECS/cbsample-libs.rpmspec
- cd $curr_working_dir
artifacts:
files:
- RPMS/x86_64/cbsample-libs*.rpm
discard-paths: yesUsing the same RPM build project that we created earlier, start a new build and set the value of the `–buildspec-override` parameter to buildspec-libs-rpm.yml .
After the build is completed successfully, check to see if the package appears in the artifact S3 bucket under the CodeBuild-RPM-Demo build project folder.
Conclusion
In this post, I have shown you how multiple buildspec files in the same source repository can be used to run multiple AWS CodeBuild build projects. I have also shown you how to provide a different buildspec file when starting the build.
For more information about AWS CodeBuild, see the AWS CodeBuild documentation. You can get started with AWS CodeBuild by using this step by step guide.
About the author
Prakash Palanisamy is a Solutions Architect for Amazon Web Services. When he is not working on Serverless, DevOps or Alexa, he will be solving problems in Project Euler. He also enjoys watching educational documentaries.