MultiCloud cost visibility with Amazon QuickSight and FOCUS

Customers with multi-cloud environments struggle to effectively track and manage their usage and costs. They access the individual billing portals of each cloud provider to gather cost information. Without a consolidated view on cloud expenditure, making data-driven decisions to optimize costs becomes difficult.

In September 2023 we introduced the Cloud Intelligence Dashboard for Azure. This offered customers a way to standardize their cost and usage visualization toolset across Amazon Web Services (AWS) and Microsoft Azure. Customers were delighted with the solution. Following its deployment, the most common request was the ability to view costs across both cloud providers within a single dashboard.

Historically, managing multi-cloud costs was challenging due to different cost and usage data models across cloud providers. In June 2024, AWS introduced Data Exports for FOCUS 1.0, which implements the FinOps Open Cost and Usage Specification (FOCUS). This new open-source standard for cloud billing data simplifies cost reporting and analysis across multiple sources, including different cloud providers, by providing consistency and standardization. Alongside this, AWS released the FOCUS Dashboard, a new Amazon QuickSight dashboard that visualizes the standardized data.

In this blog post, we will introduce the latest features of the Cloud Intelligence Dashboard for Azure. These updates enable customers to leverage the FOCUS specification and incorporate their Azure costs into the FOCUS Dashboard alongside existing AWS cost data. The result is a unified multicloud cost view.

Figure 1 – Amazon QuickSight multicloud FOCUS dashboard

We worked closely with customers like Perfios, a leading FinTech company in India, to develop the new features.

The unified dashboard for AWS and Azure costs empowered us to gain greater control over our cloud spending, optimize resource utilization, and make data-driven decisions to achieve cost savings.

Bijoy Moothedath , Director DevOps, Perfios.

In the following sections we will explain how we have extended the Cloud Intelligence Dashboard for Azure solution and how to deploy the new features.

Solution Overview

For a basic understanding of the AWS services used, review the Solution Design section in the Cloud Intelligence Dashboard for Azure workshop.

The original solution was designed to adapt to changing data formats over time. It processes data in two stages. AWS Glue first converts the data into a standard format, then Amazon Athena queries this standardized data to produce a refined dataset for QuickSight visualizations. To work with the FOCUS cost model, a new AWS Glue job was created.

The solution maintains separate storage for AWS and Azure data, avoiding their combination in a shared Amazon Simple Storage Service (Amazon S3) bucket and AWS Glue data catalog table. This approach sidesteps the complexity of merging different data structures at the storage or catalog level. Instead, Athena uses a consolidation view to join the data. This method makes it easier to add FOCUS data from other cloud providers in the future without changing how the data is stored or cataloged.

To analyze this data, two additional Athena views are used; one for detailed short-term data, and another for longer-term trends. These views are used to create QuickSight datasets, which then serve as the source for the QuickSight dashboard. This approach allows for displaying both detailed and broad cost patterns without complex changes, making it faster to get different cost insights.

The original solution handled tagging in the AWS Glue job, but this became challenging when combining datasets. It was also hard to change which tags QuickSight could see. The updated solution provides example queries to demonstrate how tags can be surfaced using Athena. This approach makes it easier to show tags from both cloud providers and reduces work when updating tags.

Finally, the solution now allows for multiple installations in the same AWS account. This means it can run alongside existing deployments. This change makes it easier for customers to switch from the Standard export to the FOCUS export. It also lets customers set up the solution more than once in a single AWS account if needed. For example, an Independent Software Vendor (ISV) that manages cloud costs for multiple customers can deploy separate instances of the solution for each client, ensuring that each customer’s cost data remains isolated and secure.

Figure 2 – Multicloud cost data integration and visualization flow

Figure 2 illustrates the multicloud cost data integration and visualization flow. This solution deploys the Azure FOCUS data bucket and Azure Data Catalog table shown in the left columns, along with numerous AWS services for data processing, storage, and visualization as detailed in the Cloud Intelligence Dashboard for Azure workshop. The other components  are deployed separately via the Data Exports for FOCUS 1.0 and FOCUS Dashboard solution.

Getting Started

Visit the Cloud Intelligence Dashboard for Azure workshop for a full setup guide and interactive FOCUS sample dashboard. The solution can be implemented using either AWS CloudFormation or Terraform as Infrastructure as Code (IaC) providers. The workshop shows you how to:

1. Setup prerequisites, including your Azure environment.
2. Upload the solutions artifacts to an S3 artifacts bucket.
3. Implement the solution by deploying AWS CloudFormation or Terraform templates.
4. Manually trigger an initial copy of data.
5. Integrate the solution with Data Exports for FOCUS 1.0 and FOCUS Dashboard.
6. Complete common administrative tasks such as changing the schedule and adding additional tags.

The code for this blog post is available in an AWS Samples Git repository, providing a valuable resource for developers and architects looking to implement or extend this solution. Costs for this lab include the following components:

• AWS resources. Processing 2GB of data per day with the default configuration costs under $100 per month at the time of this writing.
• QuickSight Enterprise. Refer to pricing.
• Microsoft Azure egress data charges. Refer to pricing.

Conclusion

In this blog post, we introduced you to the new features of the Cloud Intelligence Dashboard for Azure, which now leverages the FOCUS specification and FOCUS dashboard to provide a unified view of AWS and Azure cloud costs within a single dashboard. This solution addresses the challenges faced by customers with multicloud environments in tracking and managing their usage and costs effectively. By using the FOCUS standard, the dashboard can be easily expanded to include other cloud providers in the future, offering a scalable approach to multicloud cost management.

AWS has significantly more services, and more features within those services, than any other cloud provider, making it faster, easier, and more cost effective to move your existing applications to the cloud and build nearly anything you can imagine. Give your Microsoft applications the infrastructure they need to drive the business outcomes you want. Visit our .NET on AWS and AWS Database blogs for additional guidance and options for your Microsoft workloads. Contact us to start your migration and modernization journey today.