AWS Cloud Operations Blog

Use Amazon CloudWatch Contributor Insights for general analysis of NGINX logs

Customers build, deploy, and maintain millions of web applications on AWS and many customers deploy these applications using NGINX. The NGINX application server offers configurability, scalability, and the ability to handle millions of concurrent requests. Web application performance is key in modern enterprise infrastructure and applications. Customers leverage CloudWatch to monitor response times, uptime, and provide SLAs. Engineering teams also track several data points and metrics, such as server performance when traffic levels shift, and engineering teams know that subpar performance will degrade customer experience. This leads to underperformance in revenue, possibly affecting compliance, or risking operational downtime.

Customers can leverage Contributor Insights rules, a feature of Amazon CloudWatch, for web application log analysis. Customers can evaluate patterns in structured log events as they are streamed to CloudWatch Logs, including any custom logs sent by applications in the cloud or from on-premises servers, such as NGINX web server application logs. CloudWatch Contributor Insights enable product and engineering teams to view, investigate, and remediate issues that occur in web applications. Once configured, Contributor Insights runs continuously without manual intervention aiding operators isolate, diagnose, understand, and remediate issues during an operational event.

This blog will show you how to monitor and perform analysis of NGINX logs using CloudWatch Contributor Insights. NGINX logs provide visibility on web applications such as errors, response times, and performance of web applications. In addition to CloudWatch Contributor Insights, customers can then activate CloudWatch Metrics and CloudWatch Alarms for comprehensive system monitoring.

Prerequisites

For this walkthrough, you should have the following prerequisites:

  1. Enable NGINX server to run in an AWS account. For this example, a NodeJS web application and API will be installed, configured, and deployed using the included CloudFormation template.
  2. NGINX web application server running in an AWS account with web logs enabled in JSON format (see below for NGINX configuration).

Solution Overview

Solution Overview

The preceding figure shows how NGINX web server application logs are sent to CloudWatch through the pre-installed CloudWatch agent on the instances. All Amazon Linux 2 AMIs include CloudWatch agent. These logs are analyzed by Contributor Insights rules and report is displayed on CloudWatch dashboard.

Walkthrough

From the AWS Management Console, navigate to CloudWatch to create a log group.

Step 1: Select Create log group.

Create log group

In CloudWatch navigate to Contributor Insights. On the Contributor Insights home page, select Create a Rule.

  1. Select NGINX log group by name from the drop down.
  2. Select Custom rule for Rule type.
  3. In Contribution section – enter the unique keys ‘remote_addr’ and ‘status’ to extract remote address of the request and the status of the request. These will be visualized in the CloudWatch dashboard.
  4. Give Rule name.
  5. Select Create.

Rule definitions

Rule type

Contribution

Once you create a rule, it may take up to 5 minutes before you see any data being reported. At this point, let’s deploy the CloudFormation Stack that will create an EC2 Instance running NGINX and a Node Express Web Application w/ two API endpoints to simulate a web application and API example.

Step 2: Deploy the following CFT template:

https://nginx-app-server-cw-00.s3.us-west-2.amazonaws.com/Nginx-Web-App-Server-Express-App-Monitor-CloudWatch.yaml

Create Stack

Stack details

This will take 5–10 minutes. You can view the status of the deployment.

Cloudformation confirmation

Once the stack is deployed check the outputs tab for the NGINX URL:

NGINX URL

Step 3: The deployment takes 5–10 minutes. Once complete click on the WebsiteURL to see the following monitoring application running with two API endpoints FAILED and OK. The application is running and logging to NGINX web logs in JSON format. These logs are streamed to CloudWatch in the access_json_log log group for analysis.

Click continue to HTTP Site when prompted with the following page:

NGINX Website

Step 4: View the NGINX Web application server status and then click on /admin/health/x1 and /return-status/200 to open in new tabs (open multiple times to simulate API access):

NGINX Web application server status

Step 5: Now that the example web application and API is running, we will create a CloudWatch Dashboard to display 5XX, 4XX, 3XX response codes from the application. Then we will create CloudWatch Metrics for 5XX 4XX 3XX response codes that will be used for CloudWatch Alarms.

Creating CloudWatch Dashboard

Let’s search for log messages with 3XX response code.

3XX response code

Let’s search for log messages with 4XX response code.

4XX response code

Let’s search for log messages with 5XX response code.

5XX response code

Step 6: Now let’s create a Metric Filter that will be used for visualization on the CloudWatch dashboard and for a CloudWatch Alarm.

Create a Metric Filter

Similarly, create a new Metric Filter for 2XX, 4XX, 5XX status codes.

Step 7: Once these metrics are created, the aggregated data will aid with the reporting of the contributions in Contributor Insights. Now let’s add the Contributor Insight rule to the CloudWatch Dashboard. Select on Add to dashboard to add to existing dashboard or create a new dashboard. Here we’ll add the CI rules as well the Alarm created previously to show a single pane of glass for 5XX.

remoteAddress-status

requestURL-status

Adding to Cloudwatch dashboard1

Adding to Cloudwatch dashboard2

Summary

In this blog we explained how to stream NGINX logs from an EC2 instance to CloudWatch, parse the logs with filters, create a graph to visualize NGINX status codes, and create alarms for status codes all within a single pane of glass.

Web application/API monitoring and observability are instrumental to enterprises and their business. Product and engineering teams can view, investigate, and remediate issues that occur in web applications when web server logs are streamed to CloudWatch using the broad capabilities of CloudWatch Contributor Insights, CloudWatch Logs, CloudWatch Metrics, and CloudWatch Alarms.


About the Author

Vivek Kumar

Vivek Kumar is a Solutions Architect at AWS based out of New York. He works with some of the largest strategic AWS customers providing technical assistance and architectural guidance on various AWS services. He brings more than 2 decades of experience in software engineering and architecture roles for various large-scale enterprises.

Sukhchander Khanna

Sukhchander Khanna is a Solutions Architect at Amazon Web Services based in San Diego. He is passionate about helping customers on their journey to the cloud by providing technical guidance and best practices. He currently helps one of the largest customers in the world build scalable, secure, and cost effective solutions on AWS.

Shaileen Savage

Shaileen Savage is a Senior Product Manager at AWS, based in Seattle, Washington. As a pragmatic product manager and developer, Shaileen is passionate about helping customers and engineers find innovative observability implementations to challenges they face. What Shaileen enjoys most in AWS observability is that every customer has a unique story that requires creative solutions to solve those needs.

Matthew Barker

Matthew Barker is a Principal Product Manager at Amazon Web Services (AWS) based in Seattle, Washington. Matthew loves building products that help customers solve their observability challenges.