Optimizing streaming media workflows to reduce your carbon footprint: Part II — Codecs and implementation

In Part I of this blog post series, we introduced the Sustainability Pillar of the Amazon Web Services (AWS) Well-Architected Framework through the lens of live and on-demand video encoding workflows powered by Media Services from AWS. We provided guidance to apply best practices from the Sustainability Pillar regarding the choice of AWS Region, use of managed services, and aligning SLAs with your sustainability and carbon footprint reduction goals. In Part II of the series, we focus on choosing video codecs and configuring AWS Media Services with energy, resource efficiency, and carbon footprint reduction in mind.

Select codecs intentionally

Video compression/encoding is the process of using an algorithm to reduce the volume of data required to capture, store, and transmit video. A video codec (short for coder-decoder) is the component that compresses and decompresses video and audio data, and a video container is a multimedia file that can hold various data streams in one file. For a deeper dive into the mechanics of video compression, please review the AWS for M&E Blog Back to basics: Mechanisms used behind the scenes in video compression.

Choosing a codec involves evaluation and consideration of different trade-offs, including image quality, bitrate required for storage and transfer, cost to encode and distribute, and device requirements to decode video streams. When designing for sustainability and carbon footprint reduction, the Sustainability Pillar goes further in depth in evaluating general trade-offs in the sustainability as a non-functional requirement section. These tradeoffs can highlight areas where best practices are in tension with each other.

Within media workflows, we can look at the H.264/AVC codec as an example in evaluating trade-offs. According to third-party research by Bitmovin, H.264/AVC remains by far the most popular codec for streaming video and is supported by most modern devices. H.264/AVC is an older codec than H.265/HEVC, VP9, or AV1, and for comparison, third-party research released by the Institute for Electrical and Electronics Engineers (IEEE) reports that HEVC requires up to 50% less bitrate to achieve a similar perceived image quality as the older AVC.

Therefore, reading the Sustainability pillar best practice to minimize data movement across networks would point toward using a newer codec. However, best practices can be in tension with each other, such as the best practice to optimize impact on devices and equipment that recommends “implementing new features that are backward compatible to reduce the number of hardware replacements” required for end-users and “optimizing an application to run efficiently on devices to help to reduce their energy consumption and extend their battery life”.

Additional third-party research released by the IEEE found that hardware decoding is more efficient and can decrease the energy required in playback on devices like smart phones, tablets, smart TVs, and set-top boxes compared with software-based decoding. However, while hardware decoding is typically more energy efficient, this also necessitates a device upgrade with an embodied carbon footprint related to the materials and manufacturing, transportation, use, and end-of-life phases to consider. For a point of reference, the Reaching Net-Zero Carbon by 2040: Decarbonizing and Neutralizing the Use Phase of Connected Devices paper from Amazon Devices says the use phase “accounts for 10-15% of the overall carbon footprint of rechargeable battery-operated devices and 60-80% of the footprint of plugged-in devices”.

According to the most recent Mobile Overview Report (MOVR) from ScientiaMobile, HEVC hardware playback support on mobile devices has grown to 93.2% in recent years; however, hardware decoding support for the newer AV1 codec remained only at 3.72%. And for encoding video, according to the AWS for M&E Blog Optimizing compression settings for cost efficiencies with AWS Media Services, HEVC/H.265 “is more CPU intensive than AVC both in the encode and the decode” and AV1 “requires drastically more CPU resources than HEVC”.

Without considering all the elements of the ecosystem, a decision made to solely to optimize for data transfer and storage with a higher efficiency codec could result in increased energy required to encode/decode video, or could require duplicative outputs in legacy and modern codecs to serve all required devices. In these complex scenarios, it is up to the organization to evaluate the multiple tradeoffs and decide on the target to optimize for. This target can be based on requirements, supported playback capabilities of a specific user base, and goals of a specific organization to determine the balance of support for legacy devices while embracing efficiencies offered by modern technological advancements in codec algorithms and encoding/decoding advancements.

Implementing AWS Media Services features

AWS Media Services have features that align with Sustainability Pillar best practices to remove or refactor workload components with low or no use and minimize data movement across networks. When using the AWS Elemental MediaConvert automated ABR (adaptive bitrate) feature and performing multi-pass (Quality Optimized) encoding, you can automatically choose the number of renditions and the resolution based on the input video. MediaConvert automated ABR minimizes the total minutes of transcoded output by eliminating renditions that increase bitrate without providing increased video quality.

Automated ABR also maximizes video quality at various bitrates by employing the quality-defined variable bitrate (QVBR) rate control mode. Quality-Defined Variable Bitrate (QVBR) analyzes every macroblock, frame, and scene in the video source, and automatically allocates bits to address information differences: higher bitrate use during complex video segments, and lower bitrate use during less complex video segments. Based on tests run in the AWS for M&E blog post: Encode video in a smarter way using Automated ABR, automated ABR resulted in 33% fewer encoded outputs, 30% less total bitrate for origin storage, and 1-34% reduction in data transfer.

AWS Elemental MediaPackage also offers the potential to reduce the required live or on-demand video encoding necessary by offering a just-in-time packaging and origination product that customizes video streams for delivery in a format compatible with the device making the request. One common workflow that MediaPackage supports is taking in a single HTTP Live Streaming (HLS) input source and dynamically packaging to HLS, DASH, CMAF, and Microsoft Smooth Streaming, with or without digital rights management (DRM), on the fly.

In addition, customers selecting output formats within MediaConvert or MediaPackage that seek to reduce the total amount of encoding and packaging resources required can also evaluate whether packaging to the Common Media Application Format (CMAF) for end users is compatible with their user base and application. As described in the AWS for M&E Blog Tutorial: Creating CMAF outputs with AWS Elemental MediaConvert, “by enabling a single file format to store, process, and deliver video streams to both HLS and DASH clients using fMP4 segments, CMAF can increase efficiency and reduce complexity and cost for live and on-demand video workflows”. It is worth noting that for live streaming, MediaPackage v2 now supports shared CMAF segments across HLS and DASH manifests served from a single endpoint, which introduces additional efficiencies. And with CMAF Ingest from MediaLive, you can deliver a single set of adaptive bitrate segments for use in both DASH and HLS packages for encoding, transmission, and storage efficiency.

Tracking with proxy metrics

The Sustainability Pillar includes guidance to evaluate specific improvements with proxy metrics, saying “when you evaluate specific changes, you must also evaluate which metrics best quantify the effect of that change on the associated resource”. The QVBR benefit described previously that resulted in 33% fewer encoded outputs, 30% less total bitrate for origin storage, and 1-34% reduction in data transfer is an example of a proxy metric and improvement to track.

Following are several examples of other proxy metrics that can apply to video workloads:

For a deeper dive into understanding and implementing proxy metrics, please review the AWS Blogs Measure and track cloud efficiency with sustainability proxy metrics, Part I: What are proxy metrics?, Part II and the AWS Well-Architected Sustainability lab Turning Cost & Usage Reports into Efficiency Reports.

Conclusion

Following best practices in the Sustainability Pillar and considering sustainability as a non-functional requirement in your AWS Media Services workflows, there are often trade-offs to consider when designing an architecture to balance your organization’s business requirements and carbon footprint reduction goals. Best practices can be in tension with each other. In one of our examples, a newer video compression method that saves on network transfer and storage resources may require the user base to either use more energy decoding the video or upgrade their device altogether. This may be misaligned with the goal of reducing the carbon footprint and environmental impact of an architect’s decision. As with any trade-off, the overall costs and benefits need to be evaluated to determine the configuration that meets the needs of your organization and scenario.

For further reading on optimizing media encoding workflows for energy efficiency, see this AWS for M&E blog post: Optimize for sustainability with video encode reuse in AWS Elemental MediaLive.