AWS Partner Network (APN) Blog
Leveraging CircleCI and AWS CodeDeploy for Continuous Integration Workflows
As more organizations move toward a DevOps mindset and prioritize rapid iterations in application development and deployment, choosing an effective combination of tools has become paramount. Continuous integration platforms often function as the cornerstone of application lifecycle management processes, so getting hands-on with a product to understand how it works for your organization is a critical part of choosing the right tool.
This blog post introduces an offering from CircleCI, a member of the AWS Partner Network (APN). CircleCl is a continuous integration tool that works with AWS CodeDeploy and other deployment services to help development teams build, test, and deliver their software iteratively and efficiently. We’ve written this blog post as newcomers ourselves to the CircleCI platform, utilizing the CircleCI tools to follow our standard development methodology.. This post will also feature AWS CodeDeploy, which will deploy the code we build and test with CircleCI to our Amazon Elastic Compute Cloud (Amazon EC2) instances.
At a high level, we’ll push a code update to Github, use CircleCI to pull code from GitHub, test the code for errors, and then, by leveraging CircleCI and CodeDeploy, push the code to production.
Because we think that following along with this post and actually using CircleCI and CodeDeploy will yield the best understanding of these tools, there are a number of prerequisites that you’ll need to handle before moving forward. First, you’ll need to choose (and remember!) a region into which to deploy your application. We recommend using Northern Virginia or Oregon. Next, if you don’t have a GitHub account, create one; it’s free!
Creating a Github Account
You can create a new account here:
Once you have your GitHub account created, you will need to create a repository. For instructions on working with GitHub and creating a repository, you can reference this guide:
https://guides.github.com/activities/hello-world/
For the project, you can clone the application we will be using in the examples from this GitHub project:
https://github.com/circleci/codedeploy-demo-app/
For the repository name, you can use the same name as the demo application in order to follow the instructions in this post. If you do choose an alternate name, be sure to record that value as it will be needed later in the CircleCI configuration file.
You will want to import the demonstration application repository into your newly created repository. Scroll to the bottom of the new repository page and choose Import code. This will allow you to get your repository up to date with the demo repository.
Next, paste the following URL into the text area provided, and then choose Begin import:
https://github.com/circleci/codedeploy-demo-app.git
Once the import is complete, choose Continue to repository to view the files now available.
Your GitHub repository page will now be displayed. Copy the HTTPS clone URL that appears in the lower right of the page and save it for later use. You will need this URL to clone the code base locally on your computer in order to make necessary changes.
Creating a CircleCI Account
You will now need to create a CircleCI login. In a web browser, navigate to the following page:
Choose the link to sign up. This will take you to an SSO (single sign-on) page for GitHub. Using your GitHub login, enter your username and password information. GitHub will ask if CircleCI should have access to your repository. You will need to allow access for CircleCI to follow changes within the repository.
You can add a project to CircleCI at any time by selecting the plus (+) sign in the left navigation panel.
Once the GitHub project has been added to CircleCI, any changes committed to this demo repository will go through the full CircleCI build and test process.
For the next step, we will have to create an Amazon Simple Storage Service (Amazon S3) bucket to house the application as it migrates from CircleCI, to CodeDeploy, and finally onto our instances. Sign in to the AWS Management Console, and choose Services, Storage & Content Delivery, S3.
In the Amazon S3 console, choose Create Bucket. You will need to choose a bucket name that is unique to the region. In the Region box, choose the region you decided upon earlier. In our example, we are using US Standard, because we decided to use the US East (N. Virginia) region (us-east-1). The alternative option would be the US West (Oregon) region (us-west-2). When you have the bucket name entered and region selected, choose Create.
Make sure to record the bucket name as it will be used later when editing the CircleCI deployment configuration.
Working with AWS CloudFormation
In order for CircleCI to be able to deploy the application from GitHub to our production environment, we are going to need a production environment. We’ve provided an AWS CloudFormation template to automate the process of creating a web server cluster for the application sitting behind an HREF to Amazon Elastic Load Balancing (ELB) load balancer. The template also automates the process of installing the AWS CodeDeploy agent, making it easy to launch the application into a public environment.
To utilize this template, navigate to the following page and download the file CloudFormation-CircleCI-CodeDeploy-fontesj.template to your desktop.
https://s3.amazonaws.com/apnblog/CloudFormation-CircleCI-CodeDeploy-fontesj.template
Once you have a local copy of the template, you can view its contents by using a standard text editor or a JSON viewer/editor.
After you have reviewed the contents of the AWS CloudFormation template, log in to your AWS account to begin the deployment.
In the AWS Management Console, choose Services, and then CloudFormation as shown in the following screen illustration.
In the AWS CloudFormation console, choose Create Stack.
This will bring you to the area where you can define template details. Select a name for the stack. For our example, we will be using “CircleCI-Demo-Deploy”.
For the Source, choose Upload a template to Amazon S3, and then choose Browse. Locate the AWS CloudFormation template file that you downloaded earlier, select the file, and then choose Open.
Scroll to the bottom of the page and choose Next.
You will now need to select the AWS CloudFormation deployment parameters. Select a KeyName from the drop-down list. Make sure that you have access to the associated private key—it will be necessary if you want to log in to the instance.
The values for TagKey and TagValue will be used within the CodeDeploy configuration. For our demonstration, enter CircleCITag for TagKey and CircleCIValue for TagValue, and then choose Next.
You will now have the option to enter additional tags. These are not necessary for the purposes of this demonstration, so choose Next.
You will now have the opportunity to review the final details of the AWS CloudFormation template prior to the launch of the stack. You will need to scroll to the bottom of the page and review the information under Capabilities. There is a check box to acknowledge the creation of an AWS Identity and Access Management (IAM) policy, role, user, and profile. These permissions are needed by the CodeDeploy agent in order to facilitate the pull and push of necessary files. Select the check box to acknowledge the requirements and then choose Create.
It will take a few minutes for the stack to be created. You can review the progress within the AWS CloudFormation dashboard as shown in the following illustration:
Once the stack has finished initializing, you will be provided with an Access Key and Secret Key to be used with the CircleCI implementation. In the AWS CloudFormation console, choose the Outputs tab at the bottom of the screen as shown in the following illustration:
Importing the Code
Now that we have all of the information required for AWS CodeDeploy, open your CircleCI console, choose the project you would like to integrate, and then choose Settings.
In the settings screen, scroll to the bottom, and choose AWS CodeDeploy under Continuous Deployment.
Place the Access Key and Secret Key from your AWS CloudFormation output into the required fields in the CircleCI configuration page, and then choose Save AWS keys.
On the CircleCI configuration page, step 2 is optional, so we will go on to step 3. The CircleCI web console provides a template for the required circle.yml file in a code box. Copy the contents of the code box and paste it into a local copy of your circle.yml file.
In your copy of the circle.yml file, replace appname-1234 with the project name used in GitHub. In our example, the project name would be circleci-demo-app Next, replace the region entry with the region you used for the CloudFormation deployment. For the S3 segment, choose the bucket we created earlier. You should have this value recorded from the previous instructions. Set key_pattern to the file name convention of your choice. To avoid file naming issues, we recommend that you use the entry circleci-demo-app-{BRANCH}-{SHORT_COMMIT}
Finally, for the deployment_group setting, we are going to use the string circleci-demo-dg
The test component used at the top of the file will be discussed later.
You should now have a circle.yml file similar to the following:
Adventures in AWS CodeDeploy
Once that is complete, log back in to the AWS Management Console and choose Services, CodeDeploy. You will now create a CodeDeploy application for use with CircleCI. For the application name, use circleci-demo-app, and for the Deployment Group entry, use circleci-demo-dg.
Under Add Instances, make sure that Amazon EC2 is selected, and then enter the TagKey and TagValue that you created with AWS CloudFormation. You should have two matching instances.
For the deployment configuration, we will be choosing OneAtATime. For the Service Role ARN, we’ll use the TagKey-CodeDeployTrustRole* role that was created with AWS CloudFormation. This role will be available in the drop-down list and will have random characters after its name, because this is a role created from the AWS CloudFormation template. The additional characters will be deleted once the stack is deleted.
Once the required fields are filled in, choose Create Application at the bottom of the page.
We now have our CodeDeploy application created.
Next, we have to tell CodeDeploy where and how to deploy the application. There is a file in the demonstration application GitHub project named appspec.yml. CodeDeploy uses this file to make deployment decisions. By default, the current file will deploy the application as needed. You can make changes and test additional options by following the documentation provided here:
http://docs.thinkwithwp.com/codedeploy/latest/userguide/app-spec-ref.html
Checking in the Code
Now that we’re done with the initial setup, we can go through the complete continuous delivery process from start to finish. Using the Git client and text editor of your choice, you will now make sure that you have a copy of the demonstration application code base and start the build process.
The following examples perform these operations from the command line, but you may use a GUI if you’d like.
The first objective is to clone the repository you created earlier from GitHub. You should have the clone URL recorded and ready for use. In the example below, replace the demonstration repository with the repository URL that you created:
Switch to the project directory in order to make changes.
cd codedeploy-demo-app/
When performing a directory listing, you will notice the files already in place that we discussed previously. You will need to copy the circle.yml modified earlier into this directory. As mentioned previously, you may need to modify the region segment, and you will need to change the Amazon S3 bucket name to the bucket you created earlier.
Now that you have an updated circle.yml file copied into this directory, it is time to initialize your repository and start integrating.
Now that you have an updated circle.yml file created, it is time to initialize your repository and start integrating.
From within your repository directory, type:
git init
git add *
git commit –m “demo app initial commit”
git remote add origin
git push –u origin master
Be sure to replace the value of Repository URL with the repository URL recorded earlier.
Deploying the Code Base
This will start the process of building the application within CircleCI. From your web browser, go to your CircleCI web console and choose the project name you added to the CircleCI configuration. You should see a build running. By default, a build is triggered on CircleCI when you push new code to any branch on GitHub. Here’s an example of a successful build:
Choose the build to see more detail. You should see a Deployment section that includes actions related to AWS CodeDeploy. If you expand the Watch Deployment row, you will see the status of the build as it is sent to both of the instances configured for AWS CodeDeploy.
Let’s now go and test our web application. From the AWS Management Console, choose CloudFormation to view the status of the current stack.
Choose the name of the stack, and then choose the Outputs tab. Scroll to the end of the Outputs section, and you will see an entry for URL. This is the URL to the ELB load balancer created by CloudFormation. This load balancer routes requests for the two CodeDeploy instances, ensuring a proper balance of web requests. Choose the URL link to open the page. This will now display your demo application as deployed on two Amazon EC2 instances.
Let’s Break It!
We are now going to see what happens when a build doesn’t go as planned. From your command line directory with the repository code, edit the following file:
vim www/index.php
The top of the file should look like this:
Change the following line from:
$AppName = "Updated Demo App";
to:
$AppName =Hello World! "Demo App";
The top lines should now look like this:
After you make these changes, check the code into GitHub (the changes will auto-deploy via AWS CodeDeploy):
git add www/index.php
git commit -m "updated index page"
git push -u origin master
This will initiate a new build of the application, which will fail with a parse error.
Not to worry! Because the build failed, it was not pushed to production.
The actual tests run on CircleCI are specified in the circle.yml file. If you review the contents of the file, you will notice that the top line of the file specifies the start for the testing procedure definition.
CircleCI will attempt to automatically discover and configure the correct settings for testing the application prior to deployment. To ensure that the correct test is used, we chose to override the auto-generated value, because we have a dedicated test script written for this purpose. The success or failure of the build is then determined based upon the exit code of the script running. An exit code of 0 (zero) will result in success while non-zero will result in failure.
Now, We Fix
Using the information from the failed built alert, we have an idea of where to look for the runtime error. Let’s now fix the code and get an update into production:
vim www/index.php
Change the line from:
$AppName =Hello World! "Demo App";
to:
$AppName = "Demo App";
The top lines should now display as follows:
Once that is complete, let’s go through the check-in process once again to kick off a new build:
git add www/index.php
git commit -m "updated index page"
git push -u origin master
You can now view the progress of the new build on CircleCI. Once the build is complete, we will return to our web browser and refresh the page that is showing the Elastic Load Balancer address. We should now see an updated title for the application page.
Final Thoughts
We’ve demonstrated just a simple “Hello World” example of continuous delivery with CircleCI and AWS CodeDeploy. CircleCI can scale effortlessly to support many teams of developers pushing thousands of commits per day, and CodeDeploy can easily perform rolling deployments to a fleet of thousands of servers.
In addition, while this example used PHP and CodeDeploy, CircleCI can build and test virtually any language or tool that runs on Linux, Android, or iOS, and can deploy to any AWS service. See here for a walkthrough of deploying a Docker-based application to AWS Elastic Beanstalk, or here for an example of continuous delivery with AWS CloudFormation.
Are you having any trouble getting your build to work on CircleCI? Or do you have questions about what is possible? Contact sayhi@circleci.com and the folks at CircleCI will help you out!
If you have any questions about the content or the process outline, please don’t hesitate to reach out to me at fontesj@amazon.com.