AWS IoT Analytics Oil and Gas Customer Use Case

Our oil and gas customer wanted to deploy AWS IoT Analytics to help them:

•  Better understand their assets in the field (for example, pumps, generators, valve assemblies, and so on).
•  Derive actionable insights from their data.
•  Build a predictive maintenance solution to help reduce their costs.

By using IoT Analytics, our customer can:

• Pre-process the IoT data coming from their field assets.
• Enrich that data with various internal and external data sources.
• Provide a time-series optimized data store.
• Empower their in-house data science team to build and train machine learning models on top of data sets derived from the data store.

Business and technical challenges

Among the challenges our customer faced, the first was the inability to access their IoT data. Other business units in the enterprise owned and controlled the assets in the field. Although many had IoT data, they could not let that data leave their on-premises environment. To prove the value of pushing IoT data to the cloud, our customer used some connected assets and had data flow through an AWS IoT Greengrass core, which is software that lets you run local compute and data caching for connected devices. From the IoT Greengrass core, the data was sent to the cloud to AWS IoT Core and then ingested by IoT Analytics.

The second challenge was technical in nature and common when building IoT solutions: how to handle massive volumes of data. Our customer needed a large dataset generated from their connected assets so that they could be confident that IoT Analytics could handle volumes of streamed data moving from IoT Core through IoT Greengrass. Because time was also of the essence, our customer used historical IoT data and loaded it directly into IoT Analytics. This made it possible for the data science team to build and test their anomaly-detection models on years of data generated by assets owned by various business units. This also allowed our customer to iterate on the IoT Analytics pipeline preprocessing and enrichment steps on a large amount of data. This approach ensured our customer was getting the desired output from the pipeline process and then storing that output in their data store before having a live stream of IoT data feeding the service. It provided confidence in the pipeline output and the data schema of the data store.

Why did our customer choose AWS IoT Analytics?

Our customer chose AWS IoT Analytics for a couple of key reasons:

• The robust IoT ecosystem of services provided by AWS, including IoT Greengrass, IoT Core, and IoT Analytics.
• The integration of IoT Analytics and Amazon SageMaker. Our customer wanted the ability to build analytics models and deploy them at the edge through IoT Greengrass ML inference.

This is arguably the most compelling reason for our customer. Now, they can derive immediate value from data sitting in storage for years and bypass the long process of connecting IoT assets to the cloud, an effort that can take up to 18 months due to various security, compliance, and privacy requirements.

What has our customer learned?

Today, our customer is exploring machine learning models through our Jupyter Notebook templates. They can quickly modify the template to meet their use cases and then build and test it against their historical IoT data they can load directly in IoT Analytics. They can quickly validate their hypothesis and derive great value just from their historical IoT data. They have even gotten other business units engaged and excited about what is possible with AWS IoT Analytics. Lastly, our customer learned how challenging it can be to get IoT data from connected assets in the field and then stream that data to the cloud.

To reduce risk, our customer took a parallel approach to the project, showing value from their IoT data early on through IoT Analytics while prototyping an IoT device connectivity solution using IoT Greengrass. Through these efforts, they have rallied more support from the business, can access IoT data for analytics, and can scale with their growing volumes of data.

Expected technical and business benefits

Our customer’s goals were to validate their hypothesis that IoT data, with proper analysis, provides meaningful value to the enterprise. They also wanted to create a prototype that provides the other business units with a blueprint for connecting devices to the AWS Cloud, unlocking IoT device data from assets in the field, and moving that data to IoT Analytics for analysis.

In the near future, our customer plans to test their anomaly-detection models in IoT Analytics and deploy them at the edge using IoT Greengrass ML inference. This is for use cases where even milliseconds are not fast enough. A round-trip to the cloud and back yields too much latency or is not assured due to connectivity challenges.