Build More Sustainable AWS Workloads with the Sustainability Scanner

AWS continues to strive to lower their carbon footprint and offer energy efficient solutions for their customers via the cloud. Sustainability Scanner is an open source tool designed to help customers create a more sustainable infrastructure on AWS by evaluating your infrastructure as code against a set of sustainability best practices and suggested improvements to apply to your code. In this post we’ll review how you can use the Sustainability Scanner to improve your infrastructure.

Shared responsibility

At AWS, sustainability is a shared responsibility. AWS is responsible for the sustainability of the cloud, delivering efficient, shared infrastructure, water stewardship, and sourcing renewable power. On the other hand, customers running on AWS are responsible for the sustainability in the cloud. That includes optimizing the resource utilization for their workloads and minimizing the total resources required to be deployed for workloads.

A Non-functional Requirement

For our customers, the strategic importance of sustainability is increasing due to government regulations, as well as customer and employee demand. Sustainability has become another non-functional requirement that architects should take into account when designing and building cloud applications. During re:Invent 2021, AWS added this Sustainability pillar to the Well-Architected Framework, to address this concern. It provides guidance on how AWS can help you reduce your carbon footprint and best practices to improve the sustainability of your workloads. This can directly translate to cost savings on AWS because you pay only for what you use.

Refer to the white paper, Sustainability Pillar – AWS Well-Architected Framework to view general guidance as well as more detailed recommendations to make your workloads more sustainable.

Following all of these best practices might be difficult when you are focusing on solving your business needs. When startups are building new features or MVPs, usually functional requirements get prioritized over non-functional requirements (including sustainability). For software development, this challenge was addressed years ago by introducing static code analysis tools. These tools can analyze your code even before you run it and find potential bugs or inefficiencies in the code.

Continuing with the software building analogy – in software functional testing, there has been a consensus that the earlier testing starts, the better the overall quality of the software and the cheaper it is to fix bugs. This has been called shift-left testing, which means that testing should start earlier in the lifecycle. Now, why not use a similar approach for sustainability optimization? Why not optimize your cloud architecture for sustainability right from the beginning when it’s just been designed?

And that’s where the idea of the Sustainability Scanner comes from. Sustainability Scanner is a command line tool that can analyze your CloudFormation templates and give you recommendations on how to improve the sustainability of your cloud infrastructure even before you deploy it.

How can Sustainability Scanner be used?

Sustainability Scanner was created to fit easily into the developer workflow. It provides a sustainability score and a report with sustainability improvements that can be readily implemented in code. There are a couple of ways the tool can be incorporated in the developer cycle 1) Running the tool on your local machine and 2) As part of the Continuous Integration Continuous Delivery (CI/CD) pipeline.

Locally

The only input required for Sustainability Scanner is your CloudFormation template, which evaluates against a set of sustainability best practices and then generates the report. This way, developers can run the tool locally before deploying the template to the cloud. You can install the tool via pip package manager:

pip3 install sustainability-scanner

Or install it from source from the GitHub repository https://github.com/awslabs/sustainability-scanner.

If you install via pip, you can start scanning locally by running the command susscanner [path/to/cloudformation/template_or_templates] from the CLI.

In the gif below, you can see examples of how to interact with the tool and what the output report looks like. The sustainability report provides details on each best practice that can be incorporated, which resources you need to change to do that, and the line number where you can find that resource in your template.

As part of the CI/CD pipeline

Apart from running the scanner locally, development teams can also make it part of their CI/CD. This would automate the scanning process and every time someone pushes new code or an update to the source code repository, the sustainability checks will be run during the build phase, where you perform other tasks like style checking, security scanning, unit testing etc. And you can either fail the build or publish the sustainability report as a build output artifact, the improvements of which can then be implemented and deployed in the next run of the pipeline. The diagram below can help you understand how Sustainability Scanner would work together with your CI/CD tools. We used AWS services for CI/CD as examples; however, you are free to pick your choice of tooling.

How does it work?

The Sustainability Scanner is built upon the AWS CloudFormation Guard tool. This is a general-purpose policy-as-code evaluation tool that uses a domain-specific language (DSL) to define policies to enforce best practices for security and compliance. The sustainability scanner extends this tool by using the DSL to create policies that allow developers to check their CloudFormation templates for sustainability best practices.

Based on the defined rules, you can quantify the sustainability posture of your infrastructure and get actionable recommendations to improve it. In the tool, a number of rules are included. These rules can be disabled (and re-enabled) to fit your workload requirements. Activating or deactivating rules can be done both on a service level and individual level. This is done by updating the rules_metadata.json file. Below you can see a snippet of this file. If you want to disable recommendations related to Amazon S3, you can set line 2 to false. To disable an individual rule, in this example, ensure_all_buckets_have_lifecycle_configuration, you can set line 8 to false.

01: "s3": {
 02:    "enabled": true,
 03:    "rules": [
04:         {
05:             "rule_name": "ensure_all_buckets_have_lifecycle_configuration",
06:             "severity": "MEDIUM",
07:             "message": "Lifecycle policies will either delete data or move data between different storage classes based on data access patterns.",
 08:            "enabled": true,
09:             "links": [
10:                 "https://docs.thinkwithwp.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html"
 11:            ]
 12:        }
13:     ]
14: }

In addition to the rule definition, other information, such as the severity of the rule, a message that will be shown, and links to help get more information. The severity of a rule can be LOW, MEDIUM or HIGH; this states the importance of a rule. Rules are extendable by creating new rules under the rules directory and adding the metadata to rules_metadata.json. You can find how to write CloudFormation Guard rules on this documentation page.

With the rules and CloudFormation template as input, the CloudFormation Sustainability Scanner generates a sustainability report as output in JSON format. In this report, you get a sustainability_score – the higher the score, the more potential sustainability improvements Sustainability Scanner has identified. Higher severity rules have a greater scope for improvement e.g., failing a HIGH SEV rule will increase your score more than a LOW SEV rule. For each failed rule the resources are listed with suggestions for improvement.

You can test how sustainable your infrastructure is today by getting the tool from the CloudFormation Sustainability Scanner GitHub repository.