AWS Partner Network (APN) Blog
Ingesting Different Sensor Type Data in a Digital Twin with AWS IoT Core
By Michael Gisler, Sr. Software Engineer – Noser Engineering
By Damian McDonald, Global DevOps Architect – AWS
 |
| Noser Engineering |
 |
The Internet of Things (IoT) is a fast-growing and innovative area that enables Amazon Web Services (AWS) customers to support a wide variety of use cases. For example, customers can gain insights into manufacturing processes for analytics and optimization, collect data from thousands of sensors, and react in real time.
IoT customers can also stream sensor data into a data lake where insights can be gathered by leveraging AWS machine learning services.
Yet, taking the first steps into the world of IoT can be daunting. Where to begin? Which architecture to choose? Which of the many AWS IoT services can support a particular use case? What pitfalls should be avoided? What about costs and security?
The good news is that such a journey can be greatly enhanced, simplified, and accelerated by leveraging the IoT expertise and experience of AWS Partners such as Noser Engineering.
Noser Engineering is an AWS Select Tier Service Partner based in Switzerland that has extensive knowledge and experience in a variety of software and hardware engineering disciplines. Noser Engineering specializes in delivering IoT solutions across different industry sectors such as water control systems using Amazon Alexa, air monitoring systems, and smart factory dashboards.
In this post, you will learn how Noser Engineering helped a customer build a general digital twin platform that is capable of adapting to rapid changes in the twined building through the integration of many different sensor types.
General Digital Twin Platform for Multiple Use Cases
A digital twin is a digital representation of a physical building. It’s connected to the physical building through IoT sensors which provide the capability to comprehend, model, and analyze the performance of a building and respond quickly to changes.
Different types of IoT sensors are required to measure specific building metrics, such as a thermostat sensor to measure temperature, a barometer senor to measure humidity, electronic sensors to determine the relative position of an elevator in the building.
Noser Engineering was approached by its customer to support their initiative to develop a general digital twin platform that caters to multiple use cases and business models.
As physical buildings can change quickly over time, there is a need to ensure a digital twin can support the integration of different sensor types in a scalable way.
Noser Engineering supported the customer to overcome this challenge, ensuring new sensor types could be added on demand. The solution scales in proportion to the number of sensors added, and the resulting emitted metric data is pushed through a custom gateway which publishes the data to AWS IoT Core for ingestion.
Sensor Emission and Metric Data Ingestion
A simplified model of the general digital twin solution architecture for sensor emission and metric data ingestion is depicted below.
Figure 1 – Sensor emission and metric data ingestion.
Multiple sensors of different types are installed throughout the physical building. These sensors emit metrics which are pushed to a custom gateway solution, which is a combined hardware and software solution developed by the customer and Noser Engineering.
This custom gateway solution publishes the metric data to AWS IoT Core, where it’s ingested and persisted to Amazon Simple Storage Service (Amazon S3) for cold storage, Amazon Timestream for hot storage, Amazon Relational Database Service (Amazon RDS) for relational data persistence and analysis, and Amazon DynamoDB for schemaless data persistence.
Frontend client applications can then connect to these AWS data sources to provide a visualization of the collected sensor data.
The solution architecture considers the following key requirements:
- Device management
- Ingestion
- Storage
- Application
In addition to the primary requirements, common cross-cutting concerns such as security, deployment, logging, and monitoring must also be considered.
Managing Devices of Multiple Types and Vendors
For this customer’s digital twin project, a custom gateway solution was used to establish a secure tunnel to AWS. This gateway allows for the updating of device firmware, as well as the remote execution of a number of applications which can be run as Docker containers on the gateway. This process can be managed via AWS.
The sensor devices push their metric data to the gateway which publishes the data to AWS IoT Core for ingestion, processing, and persistence.
For this digital twin project, a large variety of sensors of different types from different vendors needed to be supported.
A recommendation when working on a project that needs to support a multi-vendor device environment is to design, develop, and use a concise domain model such as the open-source solutions Web of Things or brickschema.
Sensor Data Ingestion in AWS IoT Core
A good place to start for IoT ingestion on AWS is AWS IoT Core, which provides the main functionalities for connecting and managing IoT devices.
Other AWS IoT services include AWS IoT TwinMaker and AWS IoT SiteWise. These services can greatly speed up the development time for realizing IoT solutions.
An obvious but often neglected point is that IoT use cases should be well understood so only the necessary data to support a particular use case is ingested. For example, consider the use case of monitoring the temperature of a building. It may not be necessary for sensors to emit additional data such as barometric pressure readings or air quality metrics. These additional metrics may be useful for other use cases, but in this particular example they are not strictly required.
It’s recommended to only ingest the minimum set of data required to fulfil the needs of a specific use case. This should also include an appropriate sampling rate and change of value to ensure that unnecessary costs are avoided.
For Noser Engineering’s customer, the digital twin allows for the configuration of the sensor ingested data points via AWS to ensure the solution can be adapted as the use cases continue to evolve.
Leveraging AWS for Cold, Hot, and Analytical Storage
Although storage requirements depend on the needs of a specific use case, a general recommendation is to implement (at minimum) a hot storage and cold storage option. Hot storage and cold storage options require different databases solutions.
For this customer’s digital twin environment, a microservice architecture was chosen with multiple databases. This architecture defines a database-per-business service and brings the advantage that the ideal database can be selected and tailored for the specific service. It also avoids the pitfall of being locked into using a single database to “rule them all.”
Figure 2 – Polyglot persistence with AWS storage options.
As depicted in Figure 2, IoT data is ingested via AWS IoT Core which then invokes specific microservices depending on the type of data persistence desired. Each data persistence service provides a unique capability for the digital twin project.
Noser Engineering selected Amazon S3 as the cold storage solution for the ingested IoT sensor data. It integrates nicely with AWS analytics services such as Amazon Athena, visualization services such as Amazon QuickSight, and machine learning services such as Amazon SageMaker.
The customer’s digital twin project relies on the ability to be able to properly store IoT time-series data, and Noser Engineering recommends Amazon Timestream for hot storage. Not only does Amazon Timestream allow for the persistence of actual time-series data, but it allows for the addition of other data such as schemas and metadata to enrich the gathered IoT data.
In addition to cold and hot storage persistence, the digital twin project required the persistence of structured and unstructured data for future querying and analysis.
It was also critical to know the location of where an event happened, such as in which room of which floor is the sensor that’s sending the data located. This location data was stored outside of the hot storage in Amazon RDS for PostgreSQL and then combined with the time-series data within the application layer.
For the persistence of unstructured and schemaless data, Amazon DynamoDB is the persistence solution for unstructured data in the digital twin project.
Client Visualizations Through an Application Layer
The digital twin project provides an application layer which makes use of the IoT data, as well as other metadata, and adds the logic needed to serve the clients.
Client applications can leverage this application layer to provide rich applications such as:
- Web application for tracking and optimizing energy consumption of an office building.
- Mobile application for employees showing the availability of meeting rooms in a building.
In this customer’s case, it was important to build an open digital twin platform that could be generalized to support multiple use cases.
The customer wanted the solution to be extensible and modifiable, and Noser Engineering ensured it provided well-defined interfaces to allow extension of the digital twin and allow external consumers to benefit from the underlying AWS IoT solution.
Conclusion
In this post, you learned how Noser Engineering helped a customer to build a general digital twin platform capable of ingesting IoT data from multiple different sensor types and vendors in a scalable way.
AWS IoT Core is used to ingest the sensor data, which is then routed to microservices that persist the data to cold, hot, and analytical storage using a variety of AWS storage services. Rich client applications connect to the AWS storage services to provide visualizations of the gathered sensor data.
There is not a one-size-fits-all IoT solution or architecture. Depending on your requirements and constraints, a customized IoT solution is required. Remember that IoT is not simply sending data from things to the cloud. Rather, it’s leveraging the benefits of IoT to create use cases for customers that would not have been considered possible just a few years ago.
AWS offers a growing number of IoT services to support your IoT solution needs and accelerate your IoT journey. An excellent source of inspiration can be found in the AWS IoT Solution Repository.
Noser Engineering can accompany and support you through all stages of your IoT journey, from consulting to development and implementation to operation.
.
.
Noser Engineering – AWS Partner Spotlight
Noser Engineering is an AWS Partner based in Switzerland that has extensive knowledge and experience in a variety of software and hardware engineering disciplines.