AWS for M&E Blog

Leverage Common Media Client Data (CMCD) on AWS

Common Media Client Data (CMCD) is a valuable tool in the streaming industry, used to collect quality of service metrics across video players. However, the absence of native iOS support has hindered analytics capabilities for a significant portion of the mobile ecosystem, affecting widespread adoption. iOS 18 changes that. The latest update to HTTP Live Streaming (HLS) now integrates CMCD collection capabilities, marking a significant advancement for developers, streaming platforms, and content providers invested in iOS.

In this blog post, we explore:

  • The implementation of CMCD in iOS 18 and its impact on streaming application observability.
  • New performance metrics available through AVPlayer’s AVMetricEvent and how they offer deeper insights into playback performance and the user experience.
  • A practical guide to collecting and storing this data in Amazon Timestream and visualizing it with Amazon QuickSight using Amazon CloudFront CMCD near real-time logs.

Improve video observability with CMCD and Amazon CloudFront

In the world of video streaming, ensuring a seamless and high-quality viewing experience is paramount. However, achieving comprehensive observability across the various components involved in video delivery can be a daunting task. This is where the CMCD specification, combined with Amazon CloudFront, comes into play—unlocking new frontiers in video streaming observability for iOS devices.

Architecture visualization

Figure 1: High-level architecture.

CMCD standardized metrics for HTTP Streaming

CMCD, developed by the Consumer Technology Association (CTA) as part of the CTA-5004 specification, provides a standardized approach to collecting and transmitting key streaming metrics. The specification allows players to implement the transmission either in the header or the query string. HLS in the iOS ecosystem implementation imposes additional controls, and while it supports the majority of CMCD keys, there are exceptions.

Key points for CMCD implementation announced in the What’s new in HTTP Live Streaming 2024 follow:

  • It enables CMCD by transporting keys as HTTP request headers only.
  • It does not support: content ID (‘cid’), request maximum throughput (‘rtp’), and top bitrate (‘tb’) keys.
  • It includes one extension. The streaming format key (‘sf’) will return the value ‘h’ for an HLS stream and ‘lh’ for low-latency HLS (LL-HLS).

The deeper details: Getting started with CMCD on iOS 18

As we proceed with our explanation, we assume you have an understanding of, or previous experience with, Xcode and AVFoundation. We now focus on general guidance to enable the CMCD within the iOS ecosystem. For more information, please refer to the AVPlayer documentation. At the time of writing, iOS 18 is still in beta. Until it becomes public, you will need a beta version of Xcode.

Step 1: App components

├── ContentView.swift

├── PlayerModel.swift

└── CMCDData.swift

Step 2: Verify if the device has the version to conditionally load the CMCD in the AVPlayer. As per the documentation, client’s opt-in to sending CMCD by setting:

urlAsset.resourceLoader.sendsCommonMediaClientDataAsHTTPHeaders = true

If the version is available, then load the resource. In the PlayerModel, we define the loading conditions.

In PlayerModel.swift:

func setupPlayer(with url: URL) {
  let asset = AVURLAsset(url: url)

  if #available(iOS 18.0, *) {
    asset.resourceLoader.sendsCommonMediaClientDataAsHTTPHeaders = true
    print("[CMCD] Enabled native CMCD for iOS 18+ !!!")
  } else {
    print("[CMCD] Native CMCD not available on this iOS version")
    cmcdEnabler.enableCMCD(for: asset)
  }
}

Step 3: If necessary, customize and set the values for the CMCD data

In CMCDData.swift:

private func createCMCDData() -> [String: String] {
  let cmcdParams: [String: String] = [
    "sid": cmcdEnabler?.sessionId ?? "",
    "cid": contentId ?? "",
    "st": "v",
    "sf": "h",
    "br": String(Int((cmcdEnabler?.playbackRate ?? 1.0) * 1000)), // Playback rate in milliseconds
    "bl": String(Int((cmcdEnabler?.bufferLength ?? 0) * 1000)), // Buffer length in milliseconds
    "dl": String(Int((cmcdEnabler?.currentPlaybackTime ?? 0) * 1000)), // Current playback time in milliseconds
    "mtp": String(Int(cmcdEnabler?.measuredThroughput ?? 0)), // Measured throughput in kbps
    "ot": "av" // Assuming always audio/video content
  ]
return cmcdParams
}

HLS performance metrics with AVMetricEvent

In addition to CMCD, AVFoundation has a new AVMetricEvent feature. This introduces enhanced capabilities for media performance analysis in iOS applications. AVMetricEvent provides developers with tools for monitoring and analyzing AVPlayer performance. By providing access to various performance metrics such as startup time, stall events, buffer occupancy, and bitrate switches, it enables near real-time monitoring and detailed analysis of playback. Developers can use these metrics to identify and diagnose performance issues more effectively, facilitating targeted optimizations in media applications. AVMetricEvent offers a granular approach to understanding playback behavior, potentially leading to improvements in streaming quality and the user experience.

class AVMetricsViewModel: ObservableObject {
  @Published var metrics: [String: String] = [:]
  private var player: AVPlayer?
  private var timeObserver: Any?
  private var cancellables = Set<AnyCancellable>()

  func startMonitoring(for player: AVPlayer) {
    self.player = player

    // Observe player item changes
    player.publisher(for: \.currentItem)
      .sink { [weak self] item in
        self?.updateMetrics(for: item)
      }
      .store(in: &cancellables)

    // Add periodic time observer
    timeObserver = player.addPeriodicTimeObserver(forInterval: CMTime(seconds: 1, preferredTimescale: 1), queue: .main) { [weak self] _ in
      self?.updateMetrics(for: player.currentItem)
    }
}

Visualization on Amazon QuickSight

Large Language Models (LLMs) have recently been used to translate natural language into SQL queries, though building the infrastructure to power this functionality can be complex and expensive. Amazon Q in QuickSight reduces the undifferentiated heavy lifting by quickly integrating with various data sources and providing a natural language interface for your analytics. We will build upon the existing solution outlined in the blog to incorporate Amazon Q into Quicksight.

Once the iOS app is instrumented to add CMCD data to the chunk requests, the backend analytics pipeline is built following the steps in this blog, which guides you through putting CMCD data into Amazon Timestream. You can skip the previous steps in this blog post to set up a Grafana dashboard and instead step through the instructions that follow.

Step 1: Connect the Timestream database to Amazon QuickSight by creating a Dataset.

Navigate to Amazon QuickSight in the AWS Management Console. If this is your first time accessing QuickSight, you will need to sign-up and configure the authentication mechanism and auto discovery of resources.

Once sign-up is complete:

  • Click on Datasets in the left menu bar
  • Then, click on New dataset on the top-right
  • Select Timestream in the next screen and provide a name for your data source
  • Click on Create data source and in the next screen
  • Choose cmcd-db from the drop-down menu
    • If you changed the name of the database, use the name you gave to the database
  • Select cmcd-table from the dialog box and press Select
  • In the Finish dataset creation dialog box, choose Directly query your data and click Visualize

Step 2: Create a QuickSight Topic for your Dataset.

QuickSight topics are collections of one or more datasets that represent a subject area about which your business users can ask questions. We will create a topic that represents the CMCD data used for natural language queries.

  • Click on the QuickSight icon on the top left
  • Then, select Topics in the left menu bar
  • Click on New Topic on the top right
  • Provide a Topic name and Description
  • Click Continue
  • In the Select a dataset dialog, click on cmcd-table
  • Then, click Create

Step 3: Configure the data fields in your topic for natural language queries.

QuickSight needs context to understand the values in the Timestream database table. For example, the field cmcd_br, which represents the encoded bit-rate in CMCD, needs an update within QuickSight to reflect that it contains the bitrate information of the stream. This configuration is captured in the Topic.

  • Navigate to the Topic created previously in Step 2
  • Click on the Data tab
  • The CMCD and CloudFront fields will be listed—for each field, provide user-friendly names to be used in natural language queries

Figure 2: Define the dashboard visualization on Amazon QuickSight.

Apart from synonyms for the field, additional context can be provided in the field Description.

Figure 3: Define the dashboard visualization on Amazon QuickSight.

Step 4: Using Amazon Q in Amazon QuickSight for natural language queries.

Once all the fields are mapped use Amazon Q in QuickSight by:

  • Clicking on the button on the top of the page title “Ask a question about CMCD Metrics Topic”
  • In the dialog box on the left, type in a natural language query
    • for example, “average quality by country”
  • Amazon Q in QuickSight automatically translates the query to use the cmcd_br field and provides average bitrates by country

Figure 4: Creating new visualizations with Amazon Q and Amazon QuickSight.

To check the traffic generated by the iOS app, you can use a query such as: Number of unique users where the user-agent contains AppleCoreMedia.

Figure 5: Creating new visualizations with Amazon Q and Amazon QuickSight, filtering devices.

Conclusion

By integrating CMCD collection in iOS 18, the announcements about the HTTP Live Streaming have opened the door for developers to gain invaluable insights into session tracking, buffering, content analytics, and the overall quality of experience across all devices. This empowers the ability to optimize video experiences, streamline troubleshooting, and build rich dashboards that elevate the quality of service across the iOS ecosystem using the same collection and analytics pipeline as other user devices.

The integration of CMCD with Amazon CloudFront unlocks powerful capabilities for comprehensive video streaming observability. By seamlessly merging client-side metrics (like session tracking, buffering insights, and content analytics) with CloudFront for server-side performance data, developers gain an unprecedented level of end-to-end visibility.

Dive into the technical implementation details and start unleashing the full power of CMCD data on CloudFront today. Contact an AWS Representative to learn how we can help accelerate your business.

Further Reading:

You can learn more about CMCD and observability metrics from the following resources:

Osmar Bento

Osmar Bento

Osmar Bento is a Senior Solution Architect specializing in Direct-to-Consumer experiences for M&E, Gaming, and Sports at AWS. Osmar collaborates with customers to innovate and create tailored solutions using the AWS platform, enhancing their media and entertainment operations.

Christer Whitehorn

Christer Whitehorn

Christer is the Lead Solutions Architect specializing in AWS Media Services for the Asia Pacific region including Japan and Greater China. He has spent more than twenty years working with multiscreen video, broadcast playout and compression headend solutions.

Shamik Shah

Shamik Shah

Shamik is a Solutions Architect in the Strategic Accounts organization helping customers use AWS services effectively. When he isn't geeking out on AWS, he is experimenting with hydration percentages for sourdough bread or trying to balance on his SUP on the many lakes in Seattle!