AWS for M&E Blog

Create a live streaming CloudWatch dashboard using AWS Elemental MediaStore access logs

Customers build live streaming video workflows using AWS Elemental MediaStore as a high performant live video origin in the cloud. But when it comes to monitoring live video streams, customers can find it a little challenging to create an Amazon CloudWatch dashboard using AWS Elemental MediaStore access logs.

By default, AWS Elemental MediaStore does not publish access logs. Customers need to enable access logging on a container. MediaStore delivers access logs for objects stored in a container to Amazon CloudWatch. Access logs provide detailed access information for objects in the container, including the request type: put request to store an object, or get request to access an object, the object specified in the request, and the date and time when the request is processed.

In this blog, we provide examples of how to use Amazon CloudWatch to create a monitoring dashboard for live video streams in MediaStore, and create a dashboard to monitor live stream ingress and egress status and gain insights on potential issues.

Details about this post

Solution Overview

With this solution, you can build a live streaming CloudWatch dashboard to monitor video stream ingest and egress characteristics including transactions per second, MediaStore observed operation latencies, and HTTP operation status based on HTTP status code such as HTTP 200 OK, 400 and 500. All this is possible using MediaStore access logs.

Walkthrough

First, create a live channel using MediaStore as an origin, and start a player to play the live stream directly from MediaStore to generate egress traffic. Next, use MediaStore access log and CloudWatch to create an ingress and egress monitoring dashboard.

Prerequisites

For this walkthrough, you should have the following:

MediaStore and Live Stream Setup

First, set up a container to host a live channel. The container name is ‘mediastore_live’. The live stream follows a path structure like /<channel_name>/main/index.m3u8. In this example, the live channel path is /channel1/main/index.m3u8.

Once a MediaStore container is created, enable access logging by clicking the MediaStore name ‘mediastore_live’, then in the Info section on the next page, click “Enable access logging’ button. Next, click “Enable access logging” button in the popup window.

In order to play a live stream directly from MediaStore and enable HTTP/S access, click the “Edit policy” button in the “Container policy’” section. Copy and paste the following text in the edit text box, making sure to modify the “Resource” ARN to match your container settings.

{

"Version" : "2012-10-17",

"Statement" : [ {

"Sid" : "MediaStoreFullAccess",

"Effect" : "Allow",

"Principal" : {

"AWS" : "arn:aws:iam::xxxxxxxxxxxx:root"

},

"Action" : "mediastore:*",

"Resource" : "arn:aws:mediastore:us-west-2:xxxxxxxxxxxx:container/mediastore_live/*",

"Condition" : {

"Bool" : {

"aws:SecureTransport" : "true"

}

}

}, {

"Sid" : "PublicReadOverHttpOrHttps",

"Effect" : "Allow",

"Principal" : "*",

"Action" : [ "mediastore:GetObject", "mediastore:DescribeObject" ],

"Resource" : "arn:aws:mediastore:us-west-2:xxxxxxxxxxxx:container/mediastore_live/*",

"Condition" : {

"Bool" : {

"aws:SecureTransport" : [ "true", "false" ]

}

}

} ]

}

In the “Container CORS policy” section, click “Edit CORS Policy” and copy / paste the following text into the edit textbox, and then save.

[

{

"AllowedHeaders": [

"*"

],

"AllowedMethods": [

"GET",

"HEAD"

],

"AllowedOrigins": [

"*"

],

"ExposeHeaders": [

"*"

],

"MaxAgeSeconds": 3000

}

]

Next, configure a live stream using AWS Elemental MediaLive to AWS Elemental MediaStore. For more information, please review the blog post: How to Send Live Video to AWS Elemental MediaStore.

Stream Ingest Dashboard

One of the most important aspects of monitoring a live stream from an origin perspective is the health status of the stream ingest into the origin. MediaStore access logs provide deeper insights on live stream ingest into MediaStore containers.

MediaStore has default soft limits on Container operations (reference documentation here), such as GetObject, PutObject and DeleteObject. It is important to monitor your live stream operations and make sure the stream is not throttled by the container’s operational limits. It is important to understand the characteristics of your live stream, and perform proactive actions such as limit increase requests.

From the AWS management console, select the CloudWatch service, select Logs, and then select Insights. On the top dropdown box, search for /aws/mediastore/mediastore_live, which is the container we used, and select it. Next, copy the following text and paste it into the edit textbox, and click the Run query button. Select the Visualization tab to see the rendered graph.

fields @message

| filter (Path like "/channel1/main") and (Operation="PutObject")

| stats count(*) as TPM by bin(1m)

What we are doing in the above query is finding all the log messages with ‘/channel1/main’ in the path, with operation type of “PutObject”, and counting the total number of Put operations each minute.

Next, click the “Actions” button and select the “Add to dashboard” item, select Create new dashboard link, type “mediastore_live” in the Dashboard name text box, and click the check mark to confirm. Then, select Line widget type, type “mediastore_live” as the widget title, and click the “Add to dashboard” button at the bottom right.

From the CloudWatch dashboard, select the mediastore_live dashboard we just created. Select the “Actions” button and select “View/edit source” menu item. The Dashboard source text edit box should show up. Replace all the text with the following text block. Pay attention to the JSON structure, so you can learn how to add individual widget text blocks later. Once finished, click the “Update” button.

{

"widgets": [

{

"type": "log",

"x": 0,

"y": 0,

"width": 12,

"height": 6,

"properties": {

"query": "SOURCE '/aws/mediastore/mediastore_live' | fields @message\n| filter (Path like \"/channel1/main\") and (Operation=\"PutObject\")\n| stats count(*) as TPM by bin(1m)",

"region": "us-west-2",

"stacked": false,

"title": "mediastore_live channel1 TPM (Transaction Per Minute)",

"view": "timeSeries"

}

},

{

"type": "log",

"x": 0,

"y": 6,

"width": 12,

"height": 6,

"properties": {

"query": "SOURCE '/aws/mediastore/mediastore_live' | filter HTTPStatus like /2\\d{2}/ and Operation=\"PutObject\" | stats avg(TurnAroundTime), avg(TotalTime), percentile(TurnAroundTime, 99), percentile(TotalTime, 99) by bin(1m)",

"region": "us-west-2",

"stacked": false,

"view": "timeSeries",

"title": "PutObject Latencies (Successful Requests)"

}

}

]

}

The second widget monitors PutObject operation latencies. Higher latency spikes in the graph often indicate potential problems and should be investigated.

Next, we will create a widget to monitor HTTP status code 2xx, 4xx and 5xx on stream ingest into a MediaStore container.

Select the mediastore_live dashboard, and copy / paste the following widget block into the proper location of the JSON file. Then, click Update. Now you will have the widget showing HTTP operations with response code for 2xx, 4xx and 5xx.

{

"type": "log",

"x": 12,

"y": 12,

"width": 12,

"height": 6,

"properties": {

"query": "SOURCE '/aws/mediastore/mediastore_live' | filter (Path like \"/channel1/main\")\n| filter HTTPStatus like /2\\d{2}/ \n| filter Operation=\"GetObject\"\n| stats count() as `2xx Count` by Operation | sort `2xx Count` desc",

"region": "us-west-2",

"stacked": false,

"title": "2xx Status Count by Operation",

"view": "table"

}

},

{

"type": "log",

"x": 12,

"y": 18,

"width": 6,

"height": 6,

"properties": {

"query": "SOURCE '/aws/mediastore/mediastore_live' | filter (Path like \"/channel1/main\")\n| filter HTTPStatus like /4\\d{2}/ \n| filter Operation = \"GetObject\"\n| stats count() as `4xx Count` by Operation",

"region": "us-west-2",

"stacked": false,

"title": "4xx Status Count by Operation",

"view": "table"

}

},

{

"type": "log",

"x": 18,

"y": 18,

"width": 6,

"height": 6,

"properties": {

"query": "SOURCE '/aws/mediastore/mediastore_live' | filter (Path like \"/channel1/main\")\n| filter HTTPStatus like /5\\d{2}/ \n| filter Operation = \"GetObject\"\n| stats count() as `5xx Count` by Operation",

"region": "us-west-2",

"stacked": false,

"title": "5xx Status Count by Operation",

"view": "table"

}

}

Below is an example screen shot of a 2xx, 4xx and 5xx status code dashboard.

The above concludes our example of how to create a dashboard to monitor a live video stream status using MediaStore access logs. Now, let’s move on to create a dashboard to monitor the egress side of operations.

Stream Egress Dashboard

Since MediaStore is used as a live origin, the typical egress operation is from a CDN or multiple CDNs requesting live stream segments and manifest files as clients watch live video.

Similar to the steps in creating a stream ingress dashboard, navigate to CloudWatch, select the mediastore_live dashboard, and click “Actions”, and select “View/edit source”. Copy the following JSON text and place it to the appropriate location of the edit text box. Add a comma where needed to ensure correct syntax of the JSON file.

  {

            "type": "log",

            "x": 12,

            "y": 0,

            "width": 12,

            "height": 6,

            "properties": {

                "query": "SOURCE '/aws/mediastore/mediastore_live' | fields @message\n| filter Path like \"/channel1/main/\"\n| filter Operation=\"GetObject\"\n| stats count(*) by bin(5m)\n",

                "region": "us-west-2",

                "stacked": false,

                "title": "mediastore_live channel1 Egress TPS",

                "view": "timeSeries"

            }

        },

        {

            "type": "log",

            "x": 12,

            "y": 6,

            "width": 12,

            "height": 6,

            "properties": {

                "query": "SOURCE '/aws/mediastore/mediastore_live' | filter HTTPStatus like /2\\d{2}/ and Operation=\"GetObject\" | stats avg(TurnAroundTime), avg(TotalTime), percentile(TurnAroundTime, 99), percentile(TotalTime, 99) by bin(5m)",

                "region": "us-west-2",

                "stacked": false,

                "title": "GetObject Latencies (Successful Requests)",

                "view": "timeSeries"

            }

        },

        {

            "type": "log",

            "x": 0,

            "y": 12,

            "width": 12,

            "height": 6,

            "properties": {

                "query": "SOURCE '/aws/mediastore/mediastore_live' | filter Path like /main/\n| filter HTTPStatus like /2\\d{2}/ \n| filter Operation = \"PutObject\" or Operation=\"DeleteObject\"\n| stats count() as `2xx Count` by Operation | sort `2xx Count` desc",

                "region": "us-west-2",

                "stacked": false,

                "title": "2xx Status Count by Operation",

                "view": "table"

            }

        },

        {

            "type": "log",

            "x": 0,

            "y": 18,

            "width": 6,

            "height": 6,

            "properties": {

                "query": "SOURCE '/aws/mediastore/mediastore_live' | filter Path like /main/ \n| filter HTTPStatus like /4\\d{2}/ \n| filter Operation = \"PutObject\" or Operation=\"DeleteObject\"\n| stats count() as `4xx Count` by Operation",

                "region": "us-west-2",

                "stacked": false,

                "title": "4xx Status Count by Operation",

                "view": "table"

            }

        },

        {

            "type": "log",

            "x": 6,

            "y": 18,

            "width": 6,

            "height": 6,

            "properties": {

                "query": "SOURCE '/aws/mediastore/mediastore_live' | filter Path like /main/\n| filter HTTPStatus like /5\\d{2}/ \n| filter Operation = \"PutObject\" or Operation=\"DeleteObject\"\n| stats count() as `5xx Count` by Operation",

                "region": "us-west-2",

                "stacked": false,

                "title": "5xx Status Count by Operation",

                "view": "table"

            }

        }

Now you have a dashboard that shows live streaming status. The left side of the dashboard shows ingest related status including ingest TPM (transaction per minute), PutObject latencies, and HTTP status code information. The right side of the dashboard shows egress TPM (transaction per minute), GetObject latencies, and HTTP status code information.

What’s More

MediaStore access logs provide comprehensive information on object creation and object access. Combined with detailed CDN logs, customers can gain deeper insights on their live streaming performance, scale, and detailed client access information.

Cleaning up

To avoid incurring future charges, delete the MediaStore container, MediaLive services and MediaStore CloudWatch log groups.

Conclusion

This blog shows how to use MediaStore access logs and Amazon CloudWatch to create a live streaming dashboard. The dashboard data provides insights to live streaming performance and metrics around its operations.