AWS Partner Network (APN) Blog
Building the Business Case for Machine Learning in the Real World
By Carlos Escapa, Global AI/ML Practice Lead at AWS
By Gordon Heinrich, Partner Solutions Architect at AWS
The potential for machine learning (ML) to drive better decisions and reduce human error is becoming the new normal for organizations of all sizes and across all verticals.
According to Gartner, the global business value derived from artificial intelligence (AI) is projected to total $1.2 trillion in 2018, an increase of 70 percent from 2017, and is forecast to reach $3.9 trillion in 2022.
Many organizations feel that AI will be the biggest disruptor to their industry in the next five years, and ML will no doubt have profound effects on the workplace, as well.
Machine learning can drive incremental improvement to an existing application or disrupt an industry, and many leaders are asking if ML is right for their business.
In this post, we offer an approach to identifying real business value using machine learning. We’ll discuss how to identify and quantify which use cases are the best fit for your industry and how to derive business value.
Getting Started
Some of the questions about machine learning you may be asking are: What are the top use cases for my industry? What is the payoff? How long will it take? How much will it cost? Do I have the right resources? Do I have the right data? It it legal?
Discovering the best use case to get your organization started with machine learning requires you to identify the business processes that have the right characteristics, and then dissecting the process to understand where ML can have an impact.
As we work with Amazon Web Services (AWS) customers and AWS Partner Network (APN) Partners, common themes we often see for ML implementation include improving competitive position and customer focus, reducing human errors, and increasing productivity.
The general rule we recommend is to find high-value workflows where complex decisions are made largely on past experience or intuition, and where inferences can be used to provide a better decision. The focus should be on decision points where an error can be costly or life-threatening, or carry significant reputational risk.
How to Define the Best Use Cases
An approach that has worked for many companies is to develop a list of potential use cases and determine the business impact and force rank the list to identify top priorities.
“Our projects begin with use-case centric workshops where we work with business leaders to identify the areas where data can drive strategic growth, and we collaborate to uncover self-funding projects,” says Dr. Michael Segala, CEO of SFL Scientific, an AWS Machine Learning Competency Partner.
In this initial stage, it’s important to have executive-level agreement on the acceptance criteria that will need to be met to operationalize an ML model. Acceptance criteria are typically expressed as a confidence interval in ML inferences, and these intervals vary depending on the use case. For instance, you may want a face recognition inference to be more than 99 percent accurate, whereas 2 percent may be good enough for an ad banner click.
To dissect the use cases, we recommend an approach developed by Louis Dorard, Ph.D, author of Bootstrapping Machine Learning and General Chair of PAPIs, the first series of international conferences dedicated to real-world ML applications, and the innovations, techniques, and tools that power them.
Dr. Dorard’s Machine Learning Canvas breaks down the business process into Decisions, ML Task, Predictions, and Value Propositions.
Figure 1 – Machine Learning Canvas by Louis Dorard, Ph.D.
One of the ways we suggest improving this process is to use Amazon’s “Working Backwards” process of starting with the customers’ needs or wants, and working backwards to discover how machine learning will deliver outstanding results.
Your customer may be an end user or line-of-business owner. By working backwards, you are likely to find areas of improvement where data is underutilized, or not used at all, that were not evident from typical business analyses.
Discovering Business Value
Once the Machine Learning Canvas is completed, it’s time to calculate the business value and rank the use cases. Some of the questions raised will be: What data do I need? How do I get it?
The project’s economics will not be as attractive if you are building the infrastructure and waiting six months to capture and manage the data. Fortunately, AWS provides cost-effective and easy solutions to deploy a data lake and integrate with ML tools and services.
“The first step that we took when we created our ML practice was to create a data lake architecture and develop a Data Lake Quick Start,” says Maninder Chhabra, CEO of Cloudwick Technologies, another AWS Machine Learning Competency Partner. “Robust data ingestion processes and a data catalog are foundational pieces to enable ML development for our customers.”
Using AWS services like Amazon Simple Storage Service (Amazon S3) and Amazon SageMaker, the cost side of the use case can be easily understood and calculated. Amazon SageMaker provides a platform where data scientists can explore different algorithms and train models without needing data engineering or DevOps skills.
Before attempting to build ML models, you need to explore, evaluate, clean, and prepare your data. The data will most likely come from different sources both internally and externally.
Externally-sourced data can be used to enrich your dataset and provide a deeper set of ground truth to improve your model. A data lake is a centralized repository that allows you to store all your structured, semi-structured, and unstructured data at any scale. You can store your data as-is, without having to first structure the data and run different types of compute services to prepare your data.
Services like Amazon Athena, AWS Glue, Amazon Elastic MapReduce (EMR), and Amazon QuickSight offer easy-to-use tools to help you explore, prepare, and visualize your data.
Next, you need to envisage the behavior or phenomena you want to predict or explain, and start to put sample data through algorithms to see if machine learning can provide inferences within an acceptable confidence range.
Starting with a simpler model is often the right approach to establish a baseline, and then you can venture into more complex models or ensembles as required. Considerations at this point include precision requirements, accountability, explainability, and alignment with business and regulations.
Measurement and Success Criteria
The success criteria should be defined in the use case definition phase, providing a benchmark to evaluate performance. Different use cases will have different success metrics. For example, face recognition inference and ad banner click will have different success criteria.
Assumptions and data may also change and drift over time. The discipline of continuous improvement and learning can improve the model accuracy and increase the relevance to the business.
Testing in the real world is the best predictor of success, and Amazon SageMaker provides the tools to deploy models at scale with a single click to start generating predictions on real-time and batch data. Amazon SageMaker also includes A/B testing capabilities to test different variants of the model to achieve the best results.
Conclusion
Identifying machine learning use cases and understanding the business value can be a relatively straightforward process. Still, many organizations find it intimidating.
To facilitate this process, AWS and APN Partners offer workshops to help you dissect sample business process, brainstorm on working backwards, select the best algorithm, and determine the benefits for the new and improved business processes.
In addition, a plan to populate your data lake with the appropriate level of data governance, de-identification, and traceability can be explored. This activity often leads to a proof of concept (POC), followed by project deployment that delivers real business value.
Next Steps
Get to know our AWS Machine Learning Competency Partners to learn how they are providing solutions that help organizations solve their data challenges, enable ML and data science workflows, or offer SaaS-based capabilities that enhance end applications with machine intelligence.
APN Partners can leverage the AWS Navigate track for machine learning to build your practice step by step. The AWS Navigate Program provides a prescriptive path for APN Partners to build a specialized practice on AWS, and our Navigate tracks offer APN Partners the guidance they need to become AWS experts and deploy innovative solutions on behalf of AWS customers.
We also recommend reading this blog post by APN Machine Learning Segment Lead Kris Skrinak, titled Artificial Intelligence and Machine Learning: Going Beyond the Hype to Drive Better Business Outcomes.
Join us at AWS re:Invent
If you’re heading to AWS re:Invent 2018, join us at Global Partner Summit for a session titled “Drive Customer Value with Data-Driven Decisions” and explore how to enable data science at enterprise scale in a way that unleashes the value of corporate data, and embeds AI/ML in business processes.