Amazon Managed Grafana preview updated with new capabilities

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

Starting today, Amazon Managed Grafana is now available to all AWS customers in public preview and updated with new features and plugins. Amazon Managed Grafana manages the provisioning, setup, scaling, and maintenance of Grafana servers, eliminating the need for customers to do this themselves. Amazon Managed Grafana is seamlessly integrated with AWS Identity and Access Management (IAM), AWS Single Sign-On, and AWS Organizations.

Please join us on April 22 for a deep dive on new and upcoming Amazon Managed Grafana functionalities with the Amazon Managed Grafana service team. Click here to register.

Customers can easily provision a fully managed Grafana environment called a workspace. For information, see the Amazon Managed Grafana – Getting Started blog post.

Customers can now upgrade their AMG workspaces to Grafana Enterprise with a 30-day free trial through an AWS Marketplace subscription. They will have access to Enterprise data source plugins such as ServiceNow, Splunk, and New Relic and can receive support, training, and consultation directly from Grafana Labs.

Amazon Managed Grafana now also supports open source Grafana version 7.5, which includes new features such as:

• Switching to HTTP POST as the default method for Prometheus queries, thus enabling a much larger payload.
• Amazon CloudWatch plugin enhancements.
• AWS X-Ray service map visualization.

The new Open Distro for Elasticsearch data source plugin provides improved log data exploration and visualization for Elasticsearch data stored in Amazon Elasticsearch Service (Amazon ES) or clusters running Open Distro for Elasticsearch (an Apache 2.0-licensed distribution of Elasticsearch). Customers can also now see their Amazon Managed Grafana usage and billing information in the AWS Billing and Cost Management console.

Let’s take a deeper look at what’s new in Amazon Managed Grafana with this release.

Free trial and pricing options

Starting April 16, 2021, anyone with an AWS account can quickly create an Amazon Managed Grafana workspace. With Amazon Managed Grafana, you only pay for the number of active users each month. This means that no matter how many users are granted access to the Amazon Managed Grafana workspace, only users who signed in at least once during the month will be billed. Amazon Managed Grafana differentiates between editors and administrators who can create, edit, and configure Grafana items and features and viewers who have view-only access to a Grafana workspace. Here’s an example: All 1200 users at your company have access permissions to the Amazon Managed Grafana workspace. If only 12 users with editor permissions signed in and only 30 users with view-only permissions signed in, you will see charges for 12 editors and 30 viewers in your monthly AWS bill. Every AWS account includes a 90-day free trial for up to five free active users, with additional usage charges displayed in your AWS billing reports.

Amazon Managed Grafana and the in-place Grafana Enterprise upgrade option

Most Amazon Managed Grafana features and Grafana Enterprise functionality are available as part of the open source Grafana project. Amazon Managed Grafana also includes more than 20 data source plugins, such as Prometheus, Amazon Timestream, Amazon CloudWatch, and Azure Monitor for no additional charge. More plugins are added every month. For a current list of data sources, see Data sources in the Amazon Managed Grafana User Guide.

You can enable some Grafana data source plugins such as AppDynamics, DataDog, Dynatrace, MongoDB, New Relic, Oracle Database, ServiceNow, Splunk, Snowflake, and Wavefront by purchasing an in-place upgrade to Grafana Enterprise in your Amazon Managed Grafana workspace. The in-place upgrade is offered by Grafana Labs and includes enterprise support and training. Amazon Managed Grafana customers can activate a 30-day free trial of Grafana Enterprise from the Amazon Managed Grafana console. At the end of the trial, they can purchase the Enterprise license or contact Grafana Labs for custom pricing options. If no action is taken, Grafana Enterprise functionality is disabled at the end of the free trial.

New capabilities for querying and visualizing Open Distro for Elasticsearch and Amazon ES data

Amazon Managed Grafana now supports the new Open Distro for Elasticsearch data source plugin with Piped Processing Language (PPL) support, enabling simpler yet powerful querying and visualization capabilities for Amazon ES and Open Distro for Elasticsearch. You can use PPL to explore and find data without having to compose lengthy Elasticsearch Domain Specific Language (DSL) statements or write queries using JSON objects. With PPL, you can write queries as a set of commands delimited by pipes similar to UNIX pipes. This makes it possible to combine two or more commands as a chain, search for keywords, and feed results from one command to the next. With support for a comprehensive set of commands such as search, dedupe, sort, and more, you can now visualize your metric and log search results in Amazon Managed Grafana with automatic formatting and highlighting.

The following sample DSL query is an example of how this new capability takes effect:

GET kibana_sample_data_logs/_search{"from":0,"size":0,"timeout":"1m","query":{"bool":{"should":[{"term":{"response.keyword":{"value":"404","boost":1}}},{"term":{"response.keyword":{"value":"503","boost":1}}}],"adjust_pure_negative":true,"boost":1}},"sort":[{"_doc":{"order":"asc"}}],"aggregations":{"composite_buckets":{"composite":{"size":1000,"sources":[{"host":{"terms":{"field":"host.keyword","missing_bucket":true,"order":"asc"}}},{"response":{"terms":{"field":"response.keyword","missing_bucket":true,"order":"asc"}}}]},"aggregations":{"request_count":{"value_count":{"field":"request.keyword"}},"sales_bucket_sort":{"bucket_sort":{"sort":[{"request_count":{"order":"desc"}}],"size":10}}}}}}

source = kibana_sample_data_logs | where response='404' or response='503' | stats count(request) as request_count by host, response | sort –request_count

Figure 1: Query Workbench

Some important backend enhancements won’t change the look and feel of the plugin, but will improve your experience with alerts and aggregations.

New capabilities in Grafana 7.5

Support for X-Ray service map in the AWS X-Ray plugin

The AWS X-Ray data source plugin now supports service map visualization in Grafana 7.5. Until now, customers were able to query X-Ray traces through X-Ray filter expressions, filter traces, and visualize segment timelines in the trace data. With the new service map visualization feature, you can visualize service-to-service interactions and visually identify errors, faults, and throttles through color-coded service nodes.

Figures 2 and 3 show the service map visualization and how customers can interact with it.

Figure 2: Service map visualization

Figure 3: Node graph

Amazon CloudWatch data source enhancements

A number of improvements have been made to the Amazon CloudWatch data source. Many thanks to community contributors relvira, ilyastoli, and rubycut!

These improvements include:

• Support for the eu-south-1 Region.
• New metrics for the AWS/Timestream, AWS/RDS (RDS Proxy metrics), AWS/NetworkFirewall, AWS/GroundStation, and AWS/DDoSProtection namespaces.
• A page limit to the List Metrics API option to improve speed and reduce memory consumption.
• The ability to specify a custom endpoint in the configuration page, which can reduce request latency.

For more information, see the Grafana documentation for the Amazon CloudWatch data source.

Next-generation graph panel

The new time series graph panel is now available. It is a high-performance visualization based on the uPlot library. The time series panel uses the new panel architecture introduced in Grafana 7.0 and integrates with field options, overrides, and transformations.

Here’s a look at some of the features of the new time series graph panel.

The new interpolation mode allows you to choose between smooth, step before (which is the same as the current graph panel’s staircase option), and step after.

Figure 4: Interpolation modes

With the new soft min and max option, you can control the default axis range so that infrequent spikes in the data won’t get cut off.

Figure 5: Soft min and max

The new graph panel supports more than the two y-axes the current panel allows.

Figure 6: Multiple y-axes

There are also new display options to make your graphs shine. Together, fine-grained control over dashed lines and fill-coloring modes produce some stellar-looking charts!

Figure 7: Advanced dashed line settings and more

Variable inspector

You can use the new variable inspector feature in the dashboard to understand the variable usage.

Figure 8: Variable inspector

New transformations

The Sort by transformation allows you to sort data before sending it to the visualization. The Filter data by value transformation allows you to filter the data in Grafana and remove data points from the query result. This transformation is very useful if the data source does not natively filter by values.

Server-side expressions

This experimental feature lets you manipulate data returned from backend data source queries with math and other operations. The primary intended use case is for multi-dimensional data sources used with the upcoming next-generation alerting, but it can be used with backend data sources and visualization too.

Variable support in alert notifications

This feature allows you to inject alert query data into the rule name and the message fields of an alert notification. This provides detailed information that will help you troubleshoot more quickly.

Figure 9: Variable support in alert notifications

Pie charts

Pie charts have long been a popular feature request for Grafana. Amazon Managed Grafana now supports a new and improved pie chart panel (beta) that’s part of the Grafana visualization library. The panel is a complete rewrite, with standard styling, legends, overrides, and fields in line with the other built-in visualizations.

Figure 10: Pie charts

For more information, see the Grafana documentation about the pie chart panel.

Better format definition for trace data

The data for the trace view is sent from the data source with a clear format, which aligns with how Grafana represents other data and automatically formats trace data from data sources such as AWS X-Ray in the Grafana trace view.

Paste in SSL certificates for setting up the Postgres data source

You can now paste the SSL certification content in the UI when you set up the Postgres data source. This feature brings the Postgres data source to parity with MySQL.

Conclusion

Please join us on April 22 for a deep dive on new and upcoming Amazon Managed Grafana functionalities with the Amazon Managed Grafana service team. Click here to register.

Amazon Managed Grafana is now available in preview to all AWS customers, without the need to request preview access. Amazon Managed Grafana is available in the US East (N. Virginia) and Europe (Ireland) Regions. To get started, visit the Amazon Managed Grafana console. For more information about Amazon Managed Grafana, see the AWS Managed Grafana User Guide. For information about Amazon Managed Grafana pricing, see the pricing page.

About the authors