Monitoring MongoDB Atlas with AWS Managed Grafana and Amazon Managed Service for Prometheus

Many customers use MongoDB Atlas to store data from their modern business-critical applications. MongoDB Atlas provides highly a scalable, secure, highly-available and fully managed data platform. Operational monitoring of MongoDB Atlas clusters has a number of benefits. It helps prevent application downtime and customer disruptions, ensuring healthy functioning of MongoDB Atlas clusters. MongoDB Atlas supports different instance types each with its own CPU type, storage and memory capacities. Understanding cluster utilization is important to right-size your MongoDB cluster to your application needs.

In this blog, we describe how to use AWS Managed Service for Prometheus (AMP) and Amazon Managed Grafana (AMG) for monitoring MongoDB Atlas Clusters.

Amazon Managed Service for Prometheus (AMP) is a serverless, Prometheus-compatible monitoring service for applications that makes it easier to securely monitor application environments at scale. Amazon Managed Grafana (AMG) is a fully managed and secure data visualization service that you can use to instantly query, correlate, and visualize operational metrics, logs, and traces from multiple sources.

To ingest metrics, we leverage AWS Distro for OpenTelemetry Collector (ADOT Collector) to collect and delivery metrics to AMP. ADOT Collector is an AWS supported version of the OpenTelemetry Collector and is distributed by AWS. The main components of AWS ADOT are Collectors and Exporters. Collectors receive telemetry data from your applications and infrastructure. They can be deployed as a standalone service, sidecar containers, or embedded directly into applications. Exporters send that data to different destinations for storage, analysis, and visualization. ADOT Collect supports exporting data to various AWS services like Amazon CloudWatch, AMP, and more.

Figure 1. Solution Architecture Diagram

Figure 1 illustrates the solution architecture. MongodDB Atlas supports two ways to collect the metrics: over the public Internet via Internet Gateway or privately via VPC Peering Connection.

In this Solution Architecture, AMG connects to AMP query endpoint to retrieve Atlas metrics. We deploy ADOT Collector to an EC2 instance. The collector scrapes the metrics from a MongoDB Atlas cluster. We use AMG to visualize the metrics, while end users login to AMG using SSO provided by AWS IAM Identity Center.

Note: As of the publishing date, MongoDB Atlas supports Prometheus in scrape mode only which means Prometheus must pull metric from Atlas. AMP only supports remote write mode which means targets must push metrics to AMP. To circumvent these limitations, we use ADOT Collector to scrape metrics from Atlas and write them into AMP using a remote write endpoint. In the future, the presence of ADOT Collector may become unnecessary when AMP adds support for scraping.

Pre-requisites

• A VPC with Internet access or with VPC Peering to MongoDB Atlas cluster M10 or higher, an EC2 instance for ADOT Collector, AMP endpoint deployed into the VPC. SSO must be set up in order to log in to AMG.
• Follow the instructions to add a user that is used to log in to AMG.
  Steps to configure MongoDB Atlas to collect metrics in MongoDB Atlas

Steps to configure MongoDB Atlas to collect metrics in MongoDB Atlas

Log into your Atlas account.

• Click the “hamburger” menu icon next to the project dropdown in the upper lefthand corner of the screen.

Figure 2. MongoDB Atlas Integrations” on the Prometheus tile, and follow the guided setup flow.

• Select “Integrations” and select the Prometheus Monitoring Integration.

Figure 3. MongoDB Atlas Integration with Prometheus

• Supply username and password for login into Atlas cluster.

Figure 4. Generate Prometheus Scrape Configuration

• Select HTTP SD and click Generate Scrape Configurations, select Public Internet Targets.

Figure 5. Prometheus scrape configuration example for public internet connection

• Select “Private IP … ” if configuring for VPC peering.

Figure 6. Prometheus scrape configuration example for VPC peering connection

• Save the snippet of the ADOT configuration for later use.

Steps to install and setup ADOT

• In Secrets Manager create the following secrets:
  • MY_DISCOVERY_URL – discovery endpoint MongoDB Atlas Prometheus configuration (e.g. https://cloud.mongodb.com/prometheus/v1.0/groups/XXXXXX/discovery?targetScheme=PRIVATE)
  • MY_PROM_ENDPOINT – Prometheus remote write URL (e.g. “https://aps-workspaces.us-east-1.amazonaws.com/workspaces/ws-XXXXXXX/api/v1/remote_write”
  • MY_PROM_USER — user name from MongoDB Atlas Prometheus configuration (e.g. prom_user_XXXXX)
  • MY_KEY – password from MongoDB Atlas Prometheus configuration
  • MY_PROM_BASIC_AUTH_USER – user name from MongoDB Atlas Prometheus configuration (e.g. prom_user_XXXXX)
  • MY_PROM_BASIC_AUTH_PWD – password from MongoDB Atlas Prometheus configuration
• Provision an EC2 instance using Amazon Linux image and supplying this init script in the user_data section:

#!/bin/bash
wget https://aws-otel-collector.s3.amazonaws.com/amazon_linux/amd64/latest/aws-otel-collector.rpm 
rpm -Uvh ./aws-otel-collector.rpm
export MY_KEY="`aws secretsmanager get-secret-value --secret-id "MY_PROM_KEY" --query "SecretString" --output text`"
export MY_DISCOVERY_URL="`aws secretsmanager get-secret-value --secret-id "MY_DISCOVERY_URL" --query "SecretString" --output text`"
export MY_PROM_ENDPOINT="`aws secretsmanager get-secret-value --secret-id "MY_PROM_ENDPOINT" --query "SecretString" --output text`"
export MY_PROM_USER="`aws secretsmanager get-secret-value --secret-id "MY_PROM_USER" --query "SecretString" --output text`"
export MY_PROM_BASIC_AUTH_USER="`aws secretsmanager get-secret-value --secret-id "MY_PROM_BASIC_AUTH_USER" --query "SecretString" --output text`"
export MY_PROM_BASIC_AUTH_PWD="`aws secretsmanager get-secret-value --secret-id "MY_PROM_BASIC_AUTH_USER" --query "SecretString" --output text`"
mkdir -p /opt/aws/aws-otel-collector/etc/
cat > /opt/aws/aws-otel-collector/etc/config.yaml <<EOF
receivers:
  prometheus:
      config:
        scrape_configs:
          - job_name: "test-aws-data-lake-mongo-metrics"
            scrape_interval: 10s
            metrics_path: /metrics
            scheme : https
            basic_auth:
              username: ${MY_PROM_BASIC_AUTH_USER}
              password: ${MY_PROM_BASIC_AUTH_PWD}
            http_sd_configs:
              - url: ${MY_DISCOVERY_URL}
                refresh_interval: 60s
                basic_auth:
                  username: ${MY_PROM_USER}
                  password: ${MY_KEY}
extensions:
  sigv4auth:
    assume_role:
      sts_region: "us-east-1"
exporters:
  prometheusremotewrite:
    endpoint: "${MY_PROM_ENDPOINT}"
    auth:
      authenticator: sigv4auth
service:
  extensions: [sigv4auth]
  pipelines:
    metrics:
      receivers: [prometheus]
      exporters: [prometheusremotewrite]
EOF
sudo chown aoc /opt/aws/aws-otel-collector/etc/config.yaml
chmod 600 /opt/aws/aws-otel-collector/etc/config.yaml
/opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl -a start

• Make sure to select IAM instance profile that has access to Secrets Manager:

Figure 7. Advanced Details

• Configure a security group that allows outbound traffic to Atlas and AMP. Make sure the EC2 instance is assigned a public IP address. This public IP address will be used in Atlas networking configuration. Note the public IP address as we’ll use it later.

• Note: if you are accessing your MongoDB Atlas over public internet, Prometheus URL will look like this:
  https://cloud.mongodb.com/prometheus/v1.0/groups/XXXXXX/discovery

• To check status of ADOT use:
  sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl -a status

• To Stop ADOT:
  sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl -a stop

• To troubleshoot tail the logs:
  sudo less /opt/aws/aws-otel-collector/logs/aws-otel-collector.log

MongoDB Atlas uses IP Access List to control the access to MongoDB Atlas clusters. If you used the public route to scrape the metrics, then find the public IP address of the ADOT Collector EC2. Otherwise, in the case of VPC Peering, make sure you specify the correct CIDR block that includes private IP address of the ADOT Collector EC2 instance.

• Go to Network Access under Security, then click add IP Entry.
• Make sure you use the correct public IP or private CIDR block.

Figure 8. Atlas Network Access Configuration

Steps to set up AWS Managed Service for Prometheus

AMP is an AWS managed service that can be accessed via VPC endpoint. AMP has remote write and query endpoints. Both endpoints will automatically be created after you created the AMP workspace. Follow these steps to set up Prometheus workspace.

• Once set up, note remote write URL and query URL from Workspace summary page as we use them later.
• Update ADOT Collector config.yaml file to replace the prometheusremotewrite endpoint placeholder value.
  Steps to set up AWS Managed Grafana

Follow these steps to configure Grafana Workspace, making sure that the user you create is an Administrator.

Steps to visualize MongoDB Atlas metrics in Grafana

Once the Grafana Workspace is provisioned, log in to Grafana as the user name and click “Add your first data source” to add Prometheus.

• Give name of this data source, like AMP-1
• Paste the AMP query endpoint you noted earlier

Figure 9. AMG data source configuration

• In Auth section, select basic auth, enter username and password that defined for your SSO user.

Figure 10. AMG authentication configuration

• Click “Save & test“ at the bottom of the page. You should see a message, “Data source is working.“

Figure 11. AMG data source ready

• Next, create a dashboard and add a panel. In Data Source select your AMP workspace and select desired metrics. The metrics will be visualized in the panel.

Figure 12. Atlas metric graphic view in AMG

In this blog, we discussed how to use AWS Managed Service for Prometheus, AWS Managed Grafana, and ADOT/OTel Collector to monitor MongoDB Atlas metrics. This solution gives customers the ability to use AWS-managed services to monitor MongoDB Atlas. Check out the links below to learn more.

• Atlas on AWS
• AMG
• AMP

About the authors