Empowering Amazon’s solution architects with intuitive and performant business intelligence

In this blog post, we describe how the Solution Architect Empowerment Team (SET) moved its insights-delivery environment to Amazon QuickSight to reap better cost savings and scalability. The Solution Architect Empowerment Team (SET) is a division within the AWS Worldwide Specialist Organization (WWSO) that serves 1,600 specialist solution architects. Their main product, known as OZONE, is a data-driven empowerment framework designed to help these architects in four key areas:

1. Focusing on crucial customer activities, in line with Amazon’s customer-obsessed values.
2. Maximizing productivity and efficiency.
3. Showcasing their impact on AWS customer and service adoption.
4. Assisting leaders in making fast and impactful decisions using data.

SET develops specialized data intelligence tools in collaboration with various internal partners across different departments. These tools help the broader team enhance the customer experience by giving the solution architect leaders and stakeholders faster, more valuable insights to work with.

Challenges with disparate data, complex calculations, and performance

OZONE provides data-driven insights-as-a-service to specialist solution architects using data from various AWS sources like customer engagements, trainings, and more.

To deliver these powerful insights, OZONE needed to run on a strong business intelligence (BI) tool capable of consolidating datasets, running complex queries, and delivering dashboards with custom calculations and visualizations.

We wanted to accommodate the large, disparate data sources we were working with, especially since we needed to be able to wield those datasets (>200 GB) in running queries, analyses, and complex calculations. Integration with our cloud stack was also crucial because we needed to be able to ingest additional upstream data via the data pipeline. And we wanted to be able to use this data in more intelligent ways, such as by using machine learning to boost data analysis, detect patterns, and reduce time to insights.

In addition, we wanted to prioritize fast visual render times. Slower dashboard load times and upfront license model in the previous BI tool, made it difficult to enable self-service analytics to our broader user base.

With all of that in mind, we decided it was time to search for an alternative tool to serve our BI needs. Ideally, one that would be more cost-effective at scale, that would integrate well with our existing AWS cloud tech stack, and that would be capable of more advanced data modeling and analytics.

Reducing BI friction and enabling advanced capabilities with Amazon QuickSight

When searching for a new BI tool, we had a few key needs in mind. First, we wanted an analytics solution capable of scaling affordably—ideally while offering cost savings and improvements in other areas.

In August 2022, we decided to move our OZONE dashboards to QuickSight. Doing this allowed us to:

• Move away from user-based licensing costs.
• Build dashboards capable of more complex calculations and advanced analytics.
• Open OZONE to a broader user base.
• Take advantage of the benefits of our cloud tech stack for data pipelining (with Amazon S3, Amazon DynamoDB, AWS Glue, Amazon Redshift, and so on).
• Introduce more advanced features like Amazon QuickSight Q to enable conversational capabilities and other next-gen functionalities.
• Surface critical business insights faster and more reliably.

QuickSight’s integration with other AWS services, such as Amazon Redshift for data warehousing, enabled us to create a seamless data flow and a more efficient analytics pipeline.

We were also able to implement various business use cases for goal-setting methodologies (such as objectives and key results). We could build KPIs and metrics to measure progress against most of the strategic business objectives at the organization, team, and functional levels. These metrics include solution architect activity, salesforce opportunity values, sales cycle length, opportunity-to-win ratios, product value, solution architect engagement metrics, and more. With these insights, OZONE is able to support data models like baselining, data trends, forecasting, what-if analysis, and so on for more informed business decision-making by solution architect leadership.

OZONE Reflections and Insights dashboards

Our OZONE empowerment framework, which empowers stakeholders with the deep insights they need to make better decisions, comprises of two main dashboards: Reflections and Insights. Using QuickSight, we were able to deploy these dashboards within just 3 months.

These highly interactive dashboards allow users to drill down into visualizations, apply filters, and explore the analysis to drive deeper insights. QuickSight also makes it simple for us to customize these insights and reports, thanks to its various built-in visualization options.

OZONE Reflections provides specialist solution architects and solution architect leadership with the tools they need to gain improved visibility into how their efforts make a difference for customers. This dashboard analyzes solution architect activity at domain, team, and solution architect levels, where users can compare activity achieved with defined ambitions. We can also calculate projected pacing to understand how well the solution architects, teams, and domains are doing against the ambitions they have defined. Stakeholders can view the Domain Reflection dashboard for domain-level insights, the Team Reflection dashboard for team-level insights, and Solution Architect Reflection dashboard for solution architect–level insights.

The OZONE Insights dashboard provides KPIs and metrics that enable WWSO solution architect leaders to determine the effectiveness of their domain in achieving key business objectives.

These insights are also essential to measure solution architect impact. This is done by measuring their participation in customer engagements and other domain-specific activities that drive customer outcomes. The dashboard addresses five key business metrics: [1] the specialist solution architect’s engagement rate and their impact on opportunity monthly recurring revenue, [2] the specialist solution architect’s impact on win rate and opportunity cycle time, [3] the domain activities of top specialist solution architects, [4] which stages of an opportunity they participate in, and [5] the number of customers impacted by their participation.

OZONE’s QuickSight dashboards pull data from the Amazon Redshift cluster maintained by the AWS Finance team. By taking advantage of multiple data sources to fetch customer and account data, the dashboards can offer users information on key metrics and KPIs for customer-facing activities.

OZONE is also a data producer, since it collects inputs from specialist solution architect teams at the domain, team, and functional level via a web application portal. Our data pipeline is built using Amazon DynamoDB, AWS Glue, and Amazon S3, and is consolidated in the Amazon Redshift cluster. We maintain nightly refresh cycles for this upstream data via ETL and airflow. By doing so, we maintain that we’re working with the latest, most up-to-date dataset for creating business intelligence dashboards in QuickSight.

The following image demonstrates the technical architecture of OZONE and the data flow from the web application to QuickSight.

We’re also using QuickSight for embedded analytics. Using QuickSight’s embedded capability, we could embed an interactive dashboard into our centralized web application portal: the OZONE Intelligence Suite.

With this tool, our users can provide their customers with a single pane of glass—a consolidated data experience that helps them understand business impact and gain insight into solution architects’ performance.

Enhanced BI user experience for specialist solution architects

QuickSight has brought several significant advantages that have enhanced the user experience for our specialist solution architect leaders.

Faster insights

With QuickSight, we can deliver faster insights y reducing the time-consuming task of consolidating disparate data sources and preparing complex calculations for analysis and visualization. Thus, QuickSight resulted in significant time savings for the user. Amazon Redshift helps us build the necessary data models and connect them to QuickSight, which means we can join data models from different Amazon Redshift data lakes effortlessly.

Quicker data consolidation and user experience

The SPICE mode in OZONE accelerates the consumption of over 50 million records, and the common subexpression elimination (CSE) feature within SPICE optimizes queries for quicker loading dashboards, faster results from complex calculations, and an overall better user experience.

Greater accessibility and scalability

QuickSight’s scalability means all 1,600 specialist solution architects within the WWSO organization can access multiple datasets concurrently without impacting performance. This also positions us to expand the scope of OZONE beyond our current user base.

Faster, more natural reporting and data visualization

OZONE uses QuickSight’s autonarratives, which offer brief summaries of analyses in a narrative-ready format. This makes it simpler for specialist solution architect leaders to extract important insights for their regular business updates. These automated narratives focus on specialist solution architects’ impact and can be directly copied into business review documents and reports. OZONE’s dashboards also provide supportive visualizations of trends and comparative analysis using time series data, further illustrating these narratives.

Natural language querying, powered by Q

Through QuickSight Q, OZONE can interpret questions from specialist solution architect leaders and generate custom visualizations for quick data insights. The Q topics on underlying OZONE datasets have been trained to improve response accuracy, enabling end users to get answers to business questions not covered by the Reflections dashboard.

Overall, QuickSight has streamlined our development process, uniting critical data insights across different datasets and empowering specialist solution architect leaders to make timely decisions at scale.

Our collaboration with the QuickSight team has resulted in performance-efficient business intelligence dashboards with faster load times and optimized query times—which means we can deliver an enhanced user experience. This, in turn, has led to operational efficiencies within the WWSO solution architect organization, as specialist solution architects can measure their impact on AWS service adoption and the customer journey across various business segments.

The QuickSight dashboards look beautiful. They also load much faster than our previous tool. All in all, we’re pleased and find the dashboards very useful. We’ve already started to evangelize to all the solution architects in our team.

– WWSO SA Leader

Get started with QuickSight

Our shift to QuickSight enabled us to improve our data consolidation and insight delivery across the entire organization.

With QuickSight, we’re able to do more than ever before—at a far lower cost to serve. QuickSight also makes it possible for us to explore ever more exciting advanced features such as ML-enabled analytics and natural language querying.

To learn more about how QuickSight can help your business improve its cost-effectiveness, process efficiency, and insights delivery, visit Amazon QuickSight.

About the authors

Megha Saini is the product leader for OZONE at AWS. She works with cross functional business leaders to build scalable mechanisms and tools for achieving organization goals and customer outcomes. Her expertise is building and launching innovative data intelligence solutions for business insights, strategy, and operations.

Priya Balgi is a Data Product Leader at AWS who specializes in the development and management of data-driven products. These data products, built using Amazon QuickSight enables the leaders with insights to drive business decisions and organizational growth.

JJ Johnston is a Sr. Analytics Solution Architect at AWS. JJ helps customers leverage their data efficiently and quickly in order to stay competitive and successful in their markets. They come from a background in Financial Services, AIML Startups, and Alexa’s Data Platform, working with and advising on a multitude of data patterns giving them a wealth of experience they use to help customers leverage data for their business needs.

Tanvi Hungund is a Sr. Manager SA, Business Insights, at AWS. She is leading the team on the journey to a data driven organization that delivers insightful, actionable data products to WWSO – Solution Architects. She also is leading and driving shared partnership with the broader AWS- Sales Marketing and Global Services organization to collaborate and build products that drive Specialist Solution Architect impact thus helping Sales and Solution Architect work in concert together.