Modeling your industrial assets at scale using AWS IoT SiteWise

Introduction

Industrial and manufacturing customers increasingly rely on AWS IoT SiteWise to collect, store, organize, and monitor data from industrial equipment at scale. AWS IoT SiteWise provides an industrial data foundation for remote equipment monitoring, performance tracking, detecting abnormal equipment behavior, and support for advanced analytics use cases.

Building such as a data foundation typically involves modeling your assets and ingesting live and historical telemetry data. This may require a significant effort when addressing tens of thousands of equipment and ever-changing operations in pursuit of reducing waste and improving efficiency.

We launched three new features for AWS IoT SiteWise at re:Invent 2023 to improve your asset modeling efforts. Customers can now represent equipment components using Asset model components, promoting reusability. With Metadata bulk operations, they can model their equipment and manage changes in bulk. User-defined unique identifiers help customers achieve consistency across the organization by using their own identifiers.

In this blog post, we will examine 11 real-world customer scenarios related to asset modeling. We will share code examples to help you learn more about the new AWS IoT SiteWise features related to each scenario.

Prerequisites

Familiarity with asset modeling in AWS IoT SiteWise
An AWS account
Basic knowledge of Python

Setup the environment

First, you will configure your developer workstation with AWS credentials and verify that Python is installed. Next you will install Git, clone the code example project to your workstation, and set up the project. Finally, you will create an AWS Identity and Access Management (IAM) policy.

Create a Cloud9 environment using Amazon Linux 2 platform (recommended) or use any on-premises machine as a developer workstation
Configure AWS credentials
Verify Python 3.x is installed on your system by running python3 --version or python --version (on Windows)
Using terminal, install Git and clone the Metadata Bulk Operations Sample for AWS IoT SiteWise repository from the AWS Samples library on Github
```
sudo yum install git
git --version
git clone https://github.com/aws-samples/metadata-bulk-operations-sample-for-aws-iot-sitewise.git
```
Install required Python packages by running pip3 install -r requirements.txt
Update config/project_config.yml to provide required information for the job
- s3_bucket_name: Name of the S3 bucket where bulk definitions will be stored
- job_name_prefix: Prefix to be used for the bulk operations jobs
Create an AWS Identity and Access Management (IAM) policy with permissions that allow the exchange of AWS resources between Amazon S3, AWS IoT SiteWise, and your local machine. This will allow you to perform bulk operations.

Onboard and manage assets at scale

AWS IoT SiteWise now supports the bulk import, export, and update of industrial equipment metadata for modeling at scale. These bulk operations are accessible through new API endpoints such as CreateMetadataTransferJob, ListMetadataTransferJobs, GetMetadataTransferJob and CancelMetadataTransferJob.

With this new capability, users can bulk onboard and update assets and asset models in AWS IoT SiteWise. They can also migrate assets and asset models between different AWS IoT SiteWise accounts.

You will primarily use metadata bulk import jobs for this blog. The following diagram and steps explain the workflow involved in a metadata bulk import job.

Steps in Metadata Bulk Import Flow

Prepare a job schema JSON file for AWS IoT SiteWise resources. This would include asset models and assets, following the AWS IoT SiteWise metadata transfer job schema. Upload this file to an Amazon S3 bucket.
Make a metadata bulk import call to AWS IoT SiteWise, referencing the uploaded JSON file
AWS IoT SiteWise will import all the resources specified in the JSON file
Upon completion, AWS IoT SiteWise will return the status and a presigned Amazon S3 URL for any failures encountered
If there are failures, access the provided report to investigate and understand the root cause

You can also perform bulk operations using the console by navigating to Build → Bulk Operations. Now that you understand how metadata bulk operations work, you will see how this feature can help in the following real-world scenarios.

Scenario 1 – Onboard initial asset models & assets

During a Proof of concept (POC), our customers typically onboard a subset of their equipment to AWS IoT SiteWise. Using metadata bulk operations, you can import thousands of asset models and assets to AWS IoT SiteWise in a single import job.

For a fictitious automotive manufacturing company, import asset models and assets related to the welding lines at one of its manufacturing plants.
python3 src/import/main.py --bulk-definitions-file 1_onboard_models_assets.json

Scenario 2 – Define asset hierarchy

Once the asset models and assets are created in AWS IoT SiteWise, you can define the relationship between assets and create an asset hierarchy. This hierarchy helps users to track performance across different levels, from the equipment level to the corporate level.

Create an asset hierarchy for Sample_AnyCompany Motor manufacturing company
python3 src/import/main.py --bulk-definitions-file 2_define_asset_hierarchy.json

Scenario 3 – Associate data streams with asset properties

Our customers typically start ingesting data from their data sources such OPC UA server, even before modeling their assets. In these situations, the data ingested into SiteWise is stored in data streams that are not associated with any asset properties. Once the ingestion exercise is complete, you must associate the data streams with specific asset properties for contextualization.

Associate the data streams for Sample_Welding Robot 1 and Sample_Welding Robot 2 with corresponding asset properties.

python3 src/import/main.py --bulk-definitions-file 3_associate_data_streams_with_assets.json

In this blog, we created three separate metadata bulk import jobs. These jobs were for creating asset models and assets, defining the asset hierarchy, and associating data streams with asset properties. You can also perform all of these actions using a single metadata bulk import job.

Scenario 4 – Onboard additional assets

After demonstrating the business value during POC, the next step is to scale the solution within and across plants. This scale can include remaining assets in the same plant, and new assets from other plants.

In this scenario, you will onboard additional welding robots (#3 and #4), and a new production line (#2) from the same Chicago plant.
python3 src/import/main.py --bulk-definitions-file 4_onboard_additional_assets.json

Scenario 5 – Create new properties

You can enhance asset models to accommodate changes in data acquisition. For example, when new sensors are installed to capture additional data, you can update the corresponding asset models to reflect these changes.

Add a new property Joint 1 Temperature to Sample_Welding Robot asset model
python3 src/import/main.py --bulk-definitions-file 5_onboard_new_properties.json

Scenario 6 – Fix manual errors

Errors can occur during asset modeling especially when users manually enter information. Examples include asset serial numbers, asset descriptions, and units of measurement. To correct these errors, you can update the information with the correct details.

Correct the serial number of Sample_Welding Robot 1 asset by replacing the old serial number S1000 with S1001.
python3 src/import/main.py --bulk-definitions-file 6_fix_incorrect_datastreams.json

Scenario 7 – Relocate assets

Production line operations change for several reasons, such as process optimization, technological advancements, and equipment maintenance. As a result, some equipment may move from one production line to another. Using Metadata bulk operations, you can update the asset hierarchy to adapt to the changes in line operations.

Move Sample_Welding Robot 3 asset from Sample_Welding Line 1 to Sample_Welding Line 2.
python3 src/import/main.py --bulk-definitions-file 7_relocate_assets.json

Scenario 8 – Backup asset models and assets

AWS recommends that you take regular backups of asset models and assets. These backups can be used for disaster recovery or to roll back to a prior version. To create a backup, you can use the bulk export operation. While exporting, you can filter specific asset models and assets to include in your exported JSON file.

You will now back up the definitions of all welding robots under welding line 1. Replace <YOUR_ASSET_ID> in 6_backup_models_assets.json with the Asset ID of Sample_Welding Line 1.

python3 src/export/main.py --job-config-file 8_backup_models_assets.json

Scenario 9 – Promote asset models and assets to another environment

By using the metadata bulk export operation followed by the bulk import operation, you can promote a set of asset models and assets from one environment to another.

Promote all the asset models and assets from the development to the testing environment.
python3 src/import/main.py --bulk-definitions-file 9_promote_to_another_environment.json

Maintain consistency throughout the organization

Many industrial companies may have modeled some or most of their industrial equipment in multiple systems such as asset management systems and data historians. It is important for these companies to use common identifiers across the organization to maintain consistency.

AWS IoT SiteWise now supports the use of external ID and user-defined UUID for assets and asset models. With the external ID feature, users can map their existing identifiers with AWS IoT SiteWise UUIDs. You can interact with asset models and assets using these external IDs. The user-defined UUID feature helps users to reuse the same UUID across different environments such as development, testing, and production.

To learn about the differences between external IDs and UUIDs, refer to external IDs.

Scenario 10 – Apply external identifiers

You can apply external IDs using the AWS IoT SiteWise console, API, or metadata bulk import job. This can be done for existing asset models, or assets without any external IDs in AWS IoT SiteWise.

Apply external ID to an existing asset, for example, Sample_Welding Robot 4.
python3 src/import/main.py --bulk-definitions-file 10_apply_external_identifier.json

Promote standardization and reusability using model composition

AWS IoT SiteWise introduced support for a component model. This is an asset model type that helps industrial companies model smaller pieces of equipment and reuse them across asset models. This helps standardize and reuse common equipment components, such as motors.

For example, a CNC Lathe (asset model) is made of components such as servo motors. With this feature, a servo motor can be modeled independently as a component and reused in another asset model, such as a CNC Machining Center.

Scenario 11 – Compose asset models

You can compose asset models using the AWS IoT SiteWise console, API or metadata bulk import job.

Compose the Sample_Welding Robot asset model by independently modeling components in a welding robot, such as a robot joint.
python3 src/import/main.py --bulk-definitions-file 11_compose_models.json

Clean Up

If you no longer require the sample solution, consider removing the resources.

Run the following to remove all the asset models and assets created using this sample repository.
python3 src/remove_sitewise_resources.py --asset-external-id External_Id_Company_AnyCompany

Conclusion

In this post, we demonstrated the use of new AWS IoT SiteWise features, such as Metadata bulk operations, User-defined unique identifiers, and Asset model components. Together, these features promote standardization, reusability, and consistency across your organization, while helping you to scale and enhance your asset modeling initiatives.

About the authors

Raju Gottumukkala

is a Senior WorldWide IIoT Specialist Solutions Architect at AWS, helping industrial manufacturers in their smart manufacturing journey. Raju has helped major enterprises across the energy, life sciences, and automotive industries improve operational efficiency and revenue growth by unlocking true potential of IoT data. Prior to AWS, he worked for Siemens and co-founded dDriven, an Industry 4.0 Data Platform company.

The Internet of Things on AWS – Official Blog