Powering Intelligent Factory Operations with Cognizant’s APEx Factory Whisperer and AWS

Smart Manufacturing enables organizations to accumulate vast amounts of data from factory systems and operations. However, this data is often siloed across multiple formats and systems, making it challenging to extract meaningful insights to improve operational efficiency. Furthermore, as the experienced workforce retires, the vast knowledge about troubleshooting and resolving production issues is being lost. The traditional approach of creating custom data transformation and contextualization jobs has made it difficult for decision-makers to obtain quick and summarized information about production operations.

To enhance productivity, quality, yield, and sustainable operations, decision-makers need access to automatically contextualized, enriched, and made available on-demand data. Organizations need to democratize data discoverability to enable cross-functional collaboration. Additionally, manufacturers prefer autonomous operations where systems need to be “self-managed.” Recently, generative AI has emerged as a technology with the potential to address poor data utilization by democratizing its use through automatic contextualization and enabling collaboration between people and data. This can bridge the gap between technology investments and realizing returns on those investments.

In this blog post, we expand on a prior blog post on Cognizant’s APEx solution and demonstrate how AWS IoT SiteWise, AWS IoT TwinMaker, and Amazon Bedrock are used to provide expert guidance to mitigate critical issues in the manufacturing plant. The solution parses operational data from the manufacturing environment, alarms, historical trends, troubleshooting results, workshop manuals, standard operating procedures (SOPs), and Piping & Instrumentation (P&ID) diagrams of a factory and associated assets. We will describe how customers use “Factory Whisperer”, Cognizant’s APEx generative AI assistant solution, to increase uptime, improve quality, and reduce operating costs for manufacturing organizations.

Identifying and troubleshooting a shop floor problem with Factory Whisperer

In this example, Cognizant’s Factory Whisperer helps the plant manager identify an emerging issue, troubleshoot the issue, and then provides supporting data to substantiate the root cause identification. This is based on a real scenario from a Corn Wet Mill factory where the yield loss during oil extraction from corn feed directly impacts production output.

When the plant manager starts their day, they typically review any issues from the previous shift and inspect the dashboards for immediate issues. To quickly prioritize their first issue to tackle, they can review any alarms and available KPIs; or they can ask Factory Whisperer, “What are the highest priority issues in the Akron Plant today?”

Figure 1: Factory Whisper chat bot within APEx solution responding to question about important issues in the plant today.

Factory Whisperer responds with two issues requiring attention. The plant manager can dive into each one and may ask a question like, “What is causing the low feed pressure?”

Figure 2: Factory Whisper chat bot within APEx solution responding to question about the cause of low feed pressure.

The response provides several reasons that could cause low feed pressure and steps to resolve the issue. The response also provides artifacts such as Standard Operating Procedure documents that support this response.

Naturally, a curious plant manager will want to understand the severity of the low feed pressure, so they might ask, “Can you show me the feed pump pressure?”

Figure 3: Factory Whisper chat bot within APEx solution responding with a plot of the feed pump pressure showing the pump feed pressure trend, along with links to relevant reference videos.

A dynamic dashboard is returned with the feed pressure of the two pumps. The issue with the feed pressure or the pump itself is not evident, so the plant manager may dive deeper by inquiring “What is the health status of the K1 Feed pump?”

Figure 4: Factory Whisper chat bot within APEx solution provides user with most recent data relevant to the operation of the plant

A detailed dashboard highlighting key characteristics of the pump provides context to the plant manager. From this dashboard the plant manager observes that the health status of the pump is poor. To correct this, the plant manager must open a ticket for the plant maintenance manager; a button to initiate this workflow is available within the Factory Whisperer response. Let’s assume that the plant manager chooses to delay selecting the button to create a ticket and instead selects the button to display a digital twin of the plant, as they want to understand more about the context of this issue.

Figure 5: Factory Whisper chat bot within APEx solution shows the digital twin of the Akron plant along with critical machine metrics from pump assets from this plant.

This use case exemplifies how generative AI can streamline problem resolution, offering not just data but actionable insights for effective troubleshooting in industrial settings. This solution integrates various data sources, including real-time sensor data, historical data, manuals, and the physical layout of a plant. This holistic approach ensures that plant operators and maintenance engineers have access to all the information they need to quickly diagnose and fix issues, minimizing downtime and maximizing production efficiency. The conversational interface, dynamic visualizations, and automated recommendations in Factory Whisperer help plant managers detect and correct issues quickly, demonstrating the power of combining advanced analytics, machine learning, and real-time monitoring to optimize industrial processes and enhance operational excellence.

Technical Implementation for Factory Whisperer

Factory Whisperer uses several different AWS services to provide expert-level guidance to plant managers and maintenance teams. When a user asks Factory Whisperer about a problem, the solution uses natural language processing to understand the question. It then retrieves relevant information from a corporate knowledge base containing manuals, procedures, and past troubleshooting data, as well as real-time and historical sensor recordings. The solution uses a Retrieval-Augmented Generation (RAG) technique to combine this contextual information to provide a tailored, expert-like response that suggests steps to diagnose and fix the issue, using a large language model.

Figure 6: Reference architecture of Cognizant APEx Factory Whisperer solution

The solution uses AWS IoT SiteWise to collect and store real-time data from the manufacturing plant’s shop floor equipment and assets. AWS IoT SiteWise maps this raw operational data to the appropriate asset models. The asset models synchronize with AWS IoT TwinMaker, where the relationships between the assets and machines are described. These relationships provide the context for how machines and data related to each other and processes within the manufacturing plant. The knowledge graph feature and the entity-component concept in AWS IoT TwinMaker enable Factory Whisperer to run queries to identify how data from various machines and processes are related to the detected issue or alarm. AWS IoT TwinMaker provides a unified access layer to the data in AWS IoT SiteWise and other systems.

AWS IoT TwinMaker also provides a 3D digital twin of the manufacturing facility, helping to visually contextualize and troubleshoot issues across the facility. By overlaying sensor data measurements and highlighting areas of concern, the 3D digital twin helps plant manager better understand any challenges they observe in the data. This can help the plant manager or plant maintenance personnel validate the root cause and troubleshooting recommendation from Factory Whisperer. If additional data is required, it informs personnel where to go to collect it.

Users interact with Factory Whisperer through either text-based or voice-based inputs. Queries from the user are parsed via Amazon Lex, and the “APEx Orchestrator and RAG Service” logic, using AWS Lambda functions. This custom logic will pass the user’s question and a description of the plant hierarchy from AWS IoT TwinMaker to an Anthropic Claude LLM model running in Amazon Bedrock to parse the query. The LLM determines which machines, sensors, time frame of data, and other relevant information, such as standard operating procedures, are required to identify the root cause and provide troubleshooting steps. Based on the response, the “APEx Orchestrator and RAG Service” will query the necessary machine and alarm data over the specified time frame from the unified APIs provided by AWS IoT TwinMaker and obtain relevant snippets of user manuals, machine maintenance records, SOPs, historical troubleshooting records, etc., from Amazon Kendra. The response from these queries is provided to a separate LLM endpoint running in Amazon Bedrock to provide a root cause and the recommended troubleshooting steps. The response is then passed back to the user via text or audio for them to act or ask follow-up questions.

Conclusion

In this blog, we outlined how Cognizant combines AWS IoT SiteWise, AWS IoT TwinMaker, and Amazon Bedrock with their APEx solution to identify and troubleshoot the most critical problem in a customer’s Corn Wet Mill factory. The plant manager can identify critical issues and then focus on the most important problem. The most common reasons for the problem, data to support why these are the most likely factors, the impact of the problem, and a 3D visual demonstrating where the problem is occurring are presented. Finally, a set of troubleshooting steps are proposed. This saves the user time from root-causing and troubleshooting the problem, enabling them to focus on other aspects of the business.

For more information on how the Cognizant’s APEx solution helps manufacturers solve critical issues in your factory shop floor, visit their solution in the AWS Solution Library or in the AWS Marketplace. Start building your generative AI-based solutions using Amazon Bedrock by diving deep into the process of developing prompts to interpret questions about a manufacturing site and learning how to develop agentic workflows using Agents for Amazon Bedrock to enable executing multi-step tasks across factory systems and data sources.

Special thanks to Ashish Joshi, the Chief Architect in the IoT Center of Excellence at Cognizant Technology Solutions, and Sopan Roy, an IoT Solutions Architect in the IoT Center of Excellence at Cognizant Technology, for their contributions in developing this blog post content.