Querying industrial assets using natural language with AWS IoT SiteWise and Agents for Amazon Bedrock

Introduction

Generative AI-powered chatbots are driving productivity gains across industries by providing instant access to information from diverse data sources, accelerating decision-making, and reducing response times. In fast-paced industrial environments, process engineers, reliability experts, and maintenance personnel require quick access to accurate, real-time operational data to make informed decisions and maintain optimal performance. However, querying complex and often siloed industrial systems like SCADA, historians, and Internet of Things (IoT) platforms can be challenging and time-consuming, especially for those without specialized knowledge of how the operational data is organized and accessed.

Generative AI-powered chatbots provide natural language interfaces to access real-time asset information from disparate operational and corporate data sources. By simplifying data retrieval through conversational interactions, generative AI enables operators to spend less time gathering data and more time optimizing industrial productivity. These user-friendly chatbots empower personnel across roles with valuable operational insights, streamlining access to critical information scattered throughout operational and corporate sources.

Implementing chatbots in industrial settings requires a tool to assist a large language model (LLM) in navigating structured and unstructured data from industrial data stores to retrieve relevant information. This is where generative AI-powered agents come into play. Agents are AI systems that use an LLM to understand a problem, create a plan to solve it, and execute that plan by calling APIs, databases, or other resources. Agents act as an interface between users and complex data systems, enabling users to ask questions in natural language without needing to know the underlying data representations. For example, shop floor personnel could ask about a pump’s peak revolutions per minute (RPM) in the last hour without knowing how that data is organized. Since LLMs cannot perform complex calculations directly, agents orchestrate offloading those operations to industrial systems designed for efficient data processing. This allows end users to get natural language responses while leveraging existing data platforms behind the scenes.

In this blog post, we will guide developers through the process of creating a conversational agent on Amazon Bedrock that interacts with AWS IoT SiteWise, a service for collecting, storing, organizing, and monitoring industrial equipment data at scale. By leveraging AWS IoT SiteWise’s industrial data modeling and processing capabilities, chatbot developers can efficiently deliver a powerful solution to enable users across roles to access critical operational data using natural language.

Solution Overview

By leveraging Agents for Amazon Bedrock, we will build an agent that decomposes user requests into queries for AWS IoT SiteWise. This allows accessing operational data using natural language, without knowing query syntax or data storage. For example, a user can simply ask “What is the current RPM value for Turbine 1?” without using specific tools or writing code. The agent utilizes the contextualization layer in AWS IoT SiteWise for intuitive representations of industrial resources. See How AWS IoT SiteWise works for details on resource modeling.

From a chatbot interface, the user asks a natural language question that requires access to industrial asset data. The agent uses the OpenAPI specification (item 1) to orchestrate a plan for retrieving relevant data. It leverages an action group defining queries the agent can perform (item 2), handled by an AWS Lambda function that uses the AWS IoT SiteWise ExecuteQuery API (item 3). The agent may invoke multiple actions to execute the LLM’s plan until obtaining necessary data, e.g., querying property names, selecting the matching name, then querying recent measurements. Once provided the requested operational data, the model composes an answer to the original question (item 4).

Building the Agent

Pre-requisites

1. This solution leverages Agents for Amazon Bedrock. See Supported regions and models for a current list of supported regions and foundation models. To enable access to Anthropic Claude models, you will need to enable Model access in Amazon Bedrock. The agent described in this blog was designed and tested for Claude 3 Haiku.
2. The agent uses the SiteWise SQL engine, which requires that AWS IoT SiteWise and AWS IoT TwinMaker are integrated. Please follow these steps to create an AWS IoT TwinMaker workspace for AWS IoT SiteWise’s ExecuteQuery API.
3. The source code for this agent is available on GitHub.

To clone the repository, run the following command:

git clone https://github.com/aws-samples/aws-iot-sitewise-conversational-agent

Step 1: Deploy AWS IoT SiteWise assets

In this agent, AWS IoT SiteWise manages data storage, modeling, and aggregation, while Amazon Bedrock orchestrates multi-step actions to retrieve user-requested information. To begin, you will need real or simulated industrial assets streaming data into AWS IoT SiteWise. Follow the instructions on Getting started with AWS IoT SiteWise to ingest and model your industrial data, or use the AWS IoT SiteWise demo to launch a simulated wind farm with four turbines. Note that the instructions on step 3 and the sample questions in step 4 were prepared for the simulated wind farm and, if using your own assets, you will have to prepare your own agent instructions and test questions.

Step 2: Define the action group

Before creating an agent in Amazon Bedrock, you need to define the action group: the actions that the agent can perform. This action group will specify the individual queries the agent can make to AWS IoT SiteWise while gathering required data. An action group requires:

• An OpenAPI schema to define the API operations that the agent can invoke
• A Lambda function that will take the API operations as inputs

Step 2.1: Design the OpenAPI specification

This solution provides API operations with defined paths that describe actions the agent can execute to retrieve data from recent operations. For example, the GET /measurements/{AssetName}/{PropertyName} path takes AssetName and PropertyName as parameters. Note the detailed description that informs the agent when and how to call the actions. Developers can add relevant paths to the schema to include actions (queries) relevant to their use cases.

  "paths": {
    "/measurements/{AssetName}/{PropertyName}": {
      "get": {
        "summary": "Get the latest measurement",
        "description": "Based on provided asset name and property name, return the latest measurement available",
        "operationId": "getLatestMeasurement",
        "parameters": [
          {
            "name": "AssetName",
            "in": "path",
            "description": "Asset Name",
            "required": true,
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "PropertyName",
            "in": "path",
            "description": "Property Name",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ]

Upload the openapischema/iot_sitewise_agent_openapi_schema.json file with the OpenAPI specification to Amazon S3. Copy the bucket and path because we are going to need that in step 3.

Step 2.2: Deploy the AWS Lambda function

The agent’s action group will be defined by an AWS Lambda function. The repository comes with a template to automatically deploy a serverless application built with the Serverless Application Model (SAM). To build and deploy, clone the GitHub repository and run the following commands from the main directory, where the template.yaml file is stored.

sam build --use-container
sam deploy --guided

Follow the instructions from the prompt to complete the deployment.

The lambda_handler function will read the API path from the invocation, and will call one of the following functions depending on the request. See the example below for the action defined for the /measurements/{AssetName}/{PropertyName} path, which calls the get_latest_value function where we use the SiteWise ExecuteQuery API to select the most recent observations for a user defined property. Notice that actions can be defined to return successful and unsuccessful HTTP status codes, and that the agent can use the error code to continue the conversation and prompt the user for clarification.

def lambda_handler(event, context):
    responses = []
    try:
        api_path = event['apiPath']
        logger.info(f'API Path: {api_path}')
        body = ""
        
        if api_path == "/measurements/{AssetName}/{PropertyName}":
            asset_name = _get_named_parameter(event, "AssetName")
            property_name = _get_named_parameter(event, "PropertyName")
            try:
                body = get_latest_value(sw_client, asset_name, property_name)
            except ValueError as e:
                return {
                    'statusCode': 404,
                    'body': json.dumps({'error': str(e)})
                }

Developers interested in expanding this agent can create new methods in the Lambda function to make their queries to the IoT SiteWise ExecuteQuery API, and map those methods to new paths. The ExecuteQuery API allows developers to run complex calculations with current and historical data, which can include aggregates, value filtering, and metadata filtering.

Step 3: Build the agent with Agents for Amazon Bedrock

Go to the Amazon Bedrock console, click on Agents under Orchestration, and then click on Create Agent. Give your agent a meaningful name (e.g., industrial-agent) and select a model (e.g., Anthropic – Claude 3 Haiku).

The most important part in the agent definition are the agent instructions, which will inform the agent of what it should do and how it should interact with users. Some best practices for agent instructions include:

• Clearly defining purpose and capabilities upfront.
• Specifying tone and formality level.
• Instructing how to handle ambiguous or incomplete queries (e.g., ask for clarification).
• Guiding how to gracefully handle out-of-scope queries.
• Mentioning any specific domain knowledge or context to consider.

If you deployed the wind turbines simulation from AWS IoT SiteWise in step 1, we recommend the following instructions. Remember that agent instructions are not optional.

You are an industrial agent that helps operators get the most recent measurement available from their wind turbines. You are going to give responses in human-readable form, which means spelling out dates. Use clear, concise language in your responses, and ask for clarification if the query is ambiguous or incomplete. If no clear instruction is provided, ask for the name of the asset and the name of the property whose measurement we want to retrieve. If a query falls outside your scope, politely inform the user

Under Action Groups, select the Lambda function you created in step 3, and browse or enter the S3 URL that points to the API schema from step 2.1. Alternatively, you can directly enter the text from the API schema on the Bedrock console.

Go to Review and create.

Step 4: Test the agent

The Amazon Bedrock console allows users to test agents in a conversational setting, view the thought process behind each interaction, and utilize Advanced prompts to modify the pre-processing and orchestration templates automatically generated in the previous step.

In the Amazon Bedrock console, select the agent and click on the Test button. A chat window will pop up.

Try the agent to ask questions such as:

• What wind turbine assets are available?
• What are the properties for Turbine 1?
• What is the current value for RPM?

Notice that the agent can reason through the data from SiteWise and the chat history, often understanding the asset or property without being given the exact name. For instance, it recognizes Turbine 1 as Demo Turbine Asset 1 and RPM as RotationsPerMinute. To accomplish this, the agent orchestrates a plan: list available assets, list properties, and query based on the asset and property names stored in SiteWise, even if they do not match the user’s query verbatim.

The response given by the agent can always be tuned. By clicking the Show trace button, you can analyze the decision-making process and understand the agent’s reasoning. Additionally, you can modify the agent’s behavior by using Advanced prompts to edit the pre-processing, orchestration, or post-processing steps.

Once confident in your agent’s performance, create an alias on the Amazon Bedrock console to deploy a draft version. Under Agent details, click Create alias to publish a new version. This allows chatbot applications to programmatically invoke the agent using InvokeAgent in the AWS SDK.

Conclusion

The generative AI agent discussed in this blog enables industrial companies to develop chatbots that can interact with operational data from their industrial assets. By leveraging AWS IoT SiteWise data connectors and models, the agent facilitates the consumption of operational data, integrating generative AI with Industrial IoT workloads. This industrial chatbot can be used alongside specialized agents or knowledge bases containing corporate information, machine data, and O&M manuals. This integration provides the language model with relevant information to assist users in making critical business decisions through a single, user-friendly interface.

Call to action

Once your agent is ready, the next step is to build a user interface for your industrial chatbot. Visit this GitHub repository to learn the components of a generative AI-powered chatbot and to explore sample code.

About the Authors