Category: Analytics

In this guest post, Henrik André Olsen, Solutions Architect, discusses how he visualized AWS Config data in Amazon QuickSight dashboards with a high value for the Danish insurance company Topdanmark.  If you are an AWS Config user, you are probably already familiar with how to use the AWS Config console to access data, but it’s […]

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. In this post, I show you how to aggregate AWS Health events centrally from all accounts in your organization using AWS Organizations, AWS Lambda, and AWS Health API, and then build automation to ingest and visualize the operations data using […]

Since 2014, AWS Trusted Advisor has been providing customers with visibility into an individual AWS account and providing recommendations based on known AWS best practices. Trusted Advisor makes recommendations to help customers achieve a better security posture, control their costs, optimize application performance, design better fault tolerance, and maintain control over their AWS service limits […]

AWS Config tracks changes made to supported resources and records them as configuration items (CIs), which are JSON files delivered to an Amazon S3 bucket. These are delivered in 6-hour intervals, as configuration history files. Each file contains details about the resources that changed in that 6-hour period, for the respective resource types, such as […]

Software Asset Governance or Software Asset Management is a key component of an organization’s cyber security strategy. Different aspects of Software Asset Governance find notable mentions in renowned security frameworks and standards such as SANS CIS Critical Security Controls, NIST Special Publication 800-53, and Cloud Security Alliance’s Cloud Control Matrix. The subject goes beyond cyber-security […]

In this blog post you learn how to visualize AWS CloudTrail events, near real time, using Kibana. This solution is useful if you use an ELK (Elasticsearch, Logstash, Kibana) stack to aggregate logs from all your systems and applications, analyze these logs, and create visualizations for application and infrastructure monitoring. This solution is also useful […]

This post walks through how Comcast Corporation set up monitoring for Amazon Kinesis Video Streams. To support their business requirements and performance SLAs, they added additional aggregated metrics vended by the Kinesis Video Stream service into Amazon CloudWatch. Comcast Corporation is a global media and technology company that offers a home security solution called Xfinity […]

Many customers use Amazon EMR with Apache Spark to build scalable big data pipelines. For large-scale production pipelines, a common use case is to read complex data from a variety of sources. This data must be transformed to make it useful to downstream applications, such as machine learning pipelines, analytics dashboards, and business reports. Such […]

Some of the customers I work with provide AWS Service Catalog products to their end-users to enable self-service for launching and managing Amazon Redshift, EMR clusters or web applications at scale using AWS Auto Scaling groups. These end-users would like the ability to self-manage these resources, for example, be able to take a snapshot of […]

In a world of highly distributed applications and increasingly bespoke architectures, data monitoring tools help DevOps engineers stay abreast of ongoing system problems. This post focuses on one such feature: Amazon VPC Flow Logs. In this post, I explain how you can deliver flow log data to Amazon S3 and then use Amazon Athena to […]