Acoustic anomaly detection using Amazon Lookout for Equipment

As the modern factory becomes more connected, manufacturers are increasingly using a range of inputs (such as process data, audio, and visual) to increase their operational efficiency. Companies use this information to monitor equipment performance and anticipate failures using predictive maintenance techniques powered by machine learning (ML) and artificial intelligence (AI). Although traditional sensors built into the equipment can be informative, audio and visual inspection can also provide insights into the health of the asset. However, leveraging this data and gaining actionable insights can be highly manual and resource prohibitive.

Koch Ag & Energy Solutions, LLC (KAES) took the opportunity to collaborate with Amazon ML Solutions Lab to learn more about alternative acoustic anomaly detection solutions and to get another set of eyes on their existing solution.

The ML Solutions Lab team used the existing data collected by KAES equipment in the field for an in-depth acoustic data exploration. In collaboration with the lead data scientist at KAES, the ML Solutions Lab team engaged with an internal team at Amazon that had participated in the Detection and Classification of Acoustic Scenes and Events 2020 competition and won high marks for their efforts. After reviewing the documentation from Giri et al. (2020), the team presented some very interesting insights into the acoustic data:

• Industrial data is relatively stationary, so the recorded audio window size can be longer in duration
• Inference intervals could be increased from 1 second to 10–30 seconds.
• The sampling rates for the recorded sounds could be lowered and still retain the pertinent information

Furthermore, the team investigated two different approaches for feature engineering that KAES hadn’t previously explored. The first was an average-spectral featurizer; the second was an advanced deep learning based (VGGish network) featurizer. For this effort, the team didn’t need to use the classifier for the VGGish classes. Instead, they removed the top-level classifier layer and kept the network as a feature extractor. With this feature extraction approach, the network can convert audio input into high-level 128-dimensional embedding, which can be fed as input to another ML model. Compared to raw audio features, such as waveforms and spectrograms, this deep learning embedding is more semantically meaningful. The ML Solutions Lab team also designed an optimized API for processing all the audio files, which decreases the I/O time by more than 90%, and the overall processing time by around 70%.

Anomaly detection with Amazon Lookout for Equipment

To implement these solutions, the ML Solutions Lab team used Amazon Lookout for Equipment, a new service that helps to enable predictive maintenance. Amazon Lookout for Equipment uses AI to learn the normal operating patterns of industrial equipment and alert users to abnormal equipment behavior. Amazon Lookout for Equipment helps organizations take action before machine failures occur and avoid unplanned downtime.

Successfully implementing predictive maintenance depends on using the data collected from industrial equipment sensors, under their unique operating conditions, and then applying sophisticated ML techniques to build a custom model that can detect abnormal machine conditions before machine failures occur.

Amazon Lookout for Equipment analyzes the data from industrial equipment sensors to automatically train a specific ML model for that equipment with no ML expertise required. It learns the multivariate relationships between the sensors (tags) that define the normal operating modes of the equipment. With this service, you can reduce the number of manual data science steps and resource hours to develop a model. Furthermore, Amazon Lookout for Equipment uses the unique ML model to analyze incoming sensor data in near-real time to accurately identify early warning signs that could lead to machine failures with little or no manual intervention. This enables detecting equipment abnormalities with speed and precision, quickly diagnosing issues, taking action to reduce expensive downtime, and reducing false alerts.

With KAES, the ML Solutions Lab team developed a proof of concept pipeline that demonstrated the data ingestion steps for both sound and machine telemetry. The team used the telemetry data to identify the machine operating states and inform which audio data was relevant for training. For example, a pump at low speed has a certain auditory signature, whereas a pump at high speed may have a different auditory signature. The relationship between measurements like RPMs (speed) and the sound are key to understanding machine performance and health. The ML training time decreased from around 6 hours to less than 20 minutes when using Amazon Lookout for Equipment, which enabled faster model explorations.

This pipeline can serve as the foundation to build and deploy anomaly detection models for new assets. After sufficient data is ingested into the Amazon Lookout for Equipment platform, inference can begin and anomaly detections can be identified.

“We needed a solution to detect acoustic anomalies and potential failures of critical manufacturing machinery,” says Dave Kroening, IT Leader at KAES. “Within a few weeks, the experts at the ML Solutions Lab worked with our internal team to develop an alternative, state-of-the-art, deep neural net embedding sound featurization technique and a prototype for acoustic anomaly detection. We were very pleased with the insight that the ML Solutions Lab team provided us regarding our data and educating us on the possibilities of using Amazon Lookout for Equipment to build and deploy anomaly detection models for new assets.”

By merging the sound data with the machine telemetry data and then using Amazon Lookout for Equipment, we can derive important relationships between the telemetry data and the acoustic signals. We can learn the normal healthy operating conditions and healthy sounds in varying operating modes.

If you’d like help accelerating the use of ML in your products and services, please contact the ML Solutions Lab.

About the Authors

Michael Robinson is a Lead Data Scientist at Koch Ag & Energy Solutions, LLC (KAES). His work focuses on computer vision, acoustic, and data engineering. He leverages technical knowledge to solve unique challenges for KAES. In his spare time, he enjoys golfing, photography and traveling.

Dave Kroening is an IT Leader with Koch Ag & Energy Solutions, LLC (KAES). His work focuses on building out a vision and strategy for initiatives that can create long term value. This includes exploring, assessing, and developing opportunities that have a potential to disrupt the Operating capability within KAES. He and his team also help to discover and experiment with technologies that can create a competitive advantage. In his spare time he enjoys spending time with his family, snowboarding, and racing.

Mehdi Noori is a Data Scientist at the Amazon ML Solutions Lab, where he works with customers across various verticals, and helps them to accelerate their cloud migration journey, and to solve their ML problems using state-of-the-art solutions and technologies. Mehdi attended MIT as a postdoctoral researcher and obtained his Ph.D. in Engineering from UCF.

Xin Chen is a senior manager at Amazon ML Solutions Lab, where he leads the Automotive Vertical and helps AWS customers across different industries identify and build machine learning solutions to address their organization’s highest return-on-investment machine learning opportunities. Xin obtained his Ph.D. in Computer Science and Engineering from the University of Notre Dame.

Yunzhi Shi is a data scientist at the Amazon ML Solutions Lab where he helps AWS customers address business problems with AI and cloud capabilities. Recently, he has been building computer vision, search, and forecast solutions for customers from various industrial verticals. Yunzhi obtained his Ph.D. in Geophysics from the University of Texas at Austin.

Dan Volk is a Data Scientist at Amazon ML Solutions Lab, where he helps AWS customers across various industries accelerate their AI and cloud adoption. Dan has worked in several fields including manufacturing, aerospace, and sports and holds a Masters in Data Science from UC Berkeley.

Brant Swidler is the Technical Product Manager for Amazon Lookout for Equipment. He focuses on leading product development including data science and engineering efforts. Brant comes from an Industrial background in the oil and gas industry and has a B.S. in Mechanical and Aerospace Engineering from Washington University in St. Louis and an MBA from the Tuck school of business at Dartmouth.