IBM Consulting's Energy Data Hub on AWS

The Energy Data Hub is a real-time streaming platform that gathers, aggregates, validates, interpolates, and interprets machine and sensor data, forming the basis for operational analytics and digital twins. It is tailored for electric, gas, and water utilities, including hydrogen and renewable operations, and is built using native AWS services enhanced by IBM’s design expertise, domain knowledge, and software IP from various projects.

Energy Data Hub utilizes foundational AWS technologies, including Managed Streaming for Kafka and Apache Spark Streaming on AWS EMR. These were selected to leverage the best AWS cloud capabilities and open-source technologies, tailored to customer preferences.

The key features of Energy Data Hub are: • Supports high volume, high velocity sensor and control messages making it both an open and highly scalable IoT platform • Designed to be compatible with various database technologies, the platform supports TimeSeries databases such as AWS TimeStream and Aurora Postgres DB, enabling the creation of an optional time-correlated data historian. • Provides the data foundation for advanced analytics and AI/ML, enabling asset health scores, grid congestion forecasting, and role-specific predictive insights with what-if simulations for better planning. Leveraging AWS technologies like SageMaker, it integrates with existing analytical solutions for demand response, predictive maintenance, and energy forecasting. • Extensible, supporting both traditional and new sensor technologies and transport mechanisms, including emerging 5G. • Captures and processes streaming data from field assets and devices in near real-time, combining it with IT data for enhanced operational and situational awareness, improved operational control, and informed decision-making. • Facilitates enterprise-wide intelligent workflows and automation by integrating data from OT, IoT, and IT systems with AI/ML services, utilizing APIs exposed through AWS API Gateway. • Features a best-of-breed architecture that incorporates the finest elements of various custom-developed solutions for energy clients. This includes a rules engine for data validation to ensure data quality and completeness, along with multiple persistence technologies and adapters for seamless integration with the client's existing environment. • Built on open-source technologies to mitigate the high cost of scaling, as traditional OT platforms (e.g., historians) charge based on the number of managed endpoints, causing license costs to skyrocket with increased instrumentation of field assets and devices.