Designing a Single Pane of Glass for Securing your Globally Deployed IoT-Workload

Introduction

Companies are investing in large-scale Internet of Things (IoT) projects and deploying global scale IoT platform such as Deutsche Bahn or Carrier. Enterprises are looking for a solution that offers a multi-tenant Single Pane of Glass Device Lifecycle Management (DLM) which caters to both IT and OT operations.

In this blog we will focus on giving perspective guidance on how to architect a multi-tenant Single Pane of Glass IoT Platform for cyber-security posture. Enterprises of all shapes and sizes from different industry can benefit from such platform. From an IT point-of-view this platform would standardize enterprise IoT related cyber-security features such as device on-boarding, visibility and governance. From an OT point of view the platform would accelerate time to production since all the heavy lifting (account management, workload management, security etc.) is baked into the platform from day one.

AWS Services

In this guidance blog we will be referencing several AWS Services. These services are integral parts of the reference architectures and best practices for the Single Pane of Glass approach.

AWS Organizations is an account management service that enables you to consolidate multiple AWS accounts into an organization that you create and centrally manage. AWS Organizations includes account management and consolidated billing capabilities that enable you to better meet the budgetary, security, and compliance needs of your business. For more information go to AWS Organizations.

AWS IoT Core lets you connect billions of IoT devices and route trillions of messages to AWS services without managing infrastructure. AWS IoT Core supports a number of communication protocols and connectivity methods. For more information go to AWS IoT Core.

AWS IoT Device Defender is natively integrated with AWS IoT Core, and it is a security service that allows you to audit the configuration of your devices, monitor connected devices to detect abnormal behavior, and mitigate security risks. For more information about go to AWS IoT Device Defender.

AWS Security Hub is a cloud security posture management service that performs security best practice checks, aggregates alerts, and enables automated remediation. It provides you with a comprehensive view of your security state in AWS and helps you check your environment against security industry standards and best practices. For more information go to AWS Security Hub.

Amazon EventBridge is Amazon EventBridge is a server-less event bus service that you can use to connect your applications with data from a variety of sources. EventBridge delivers a stream of real-time data from your applications, software as a service (SaaS) applications, and AWS services to targets such as AWS Lambda functions, HTTP invocation endpoints using API destinations, or event buses in other AWS accounts. For more information go to Amazon EventBridge.

Use Case Introduction

Figure 1. Architecture introducing the high level solution

Figure 1 shows a high level view on the issue we want to solve. We are displaying three example workloads: Refinery, Fuel Cells and Lubricants. Each of these use cases has their own IoT deployment in a distinct AWS region. Two different personas are displayed within the architecture: The Business User on a per use case level as well as the IoT Platform Administrators. Each Business User Persona needs access to their own IoT workload deployment. In this case the Refinery Business User needs the authentication as well as authorization to access the Refinery Deployments. The Lubricants Business User needs access to the Lubricants IoT workload, but not others like the Refinery. On the other hand we have the IoT Platform Admins that need access to all the workloads, no matter the region, account or use case. Additionally, the IoT Security Admin also will need to access and gain visibility into the security posture of all workloads deployed and be aware of e.g. expiring certificates.

Tenancy Model

Figure 2. Illustration the tenancy type for this architecture

For the above-mentioned use-case we are will lay a foundation of our design by employing a style of tenancy which provides complete isolation of the IoT workloads. The highly isolated tenancy design provides cost, data and workload isolation. This allows easier management of resources deployed in an AWS account and setups the foundation for isolated IoT workloads for our OT business users while providing global insight for the IT Platform admins and security personas. This tenancy style also reduces the blast radius from a security point of view since the business users and their devices are accessible through their own tenant workload. This type of tenancy comes with its own challenges related to meshing of the tenants, cost visibility and implementing Single Pane of Glass IoT platform for global device management.

Control, Data and Edge Plane

Figure 3. Visualization of Control, Data and Edge Plane

From the above illustrations in figure 1 & 2 we can compartmentalize the components in such a way where all control related use-cases are achieved through a single common interface called the IoT platform. This component serves as the Single Pane of Glass IoT device management portal for all personas. Since this component is control related, we can house this component in a conceptual boundary known as “Control Plane”.

The distinct tenant specific workloads component is specified as “IoT workload”. Since these are isolated workloads where devices connect to and send their telemetry these isolated tenant specific components can be housed in a conceptual boundary known as the “Data/Telemetry plane“. All devices managed by individual business deployed across their businesses can be housed in a conceptual boundary known as the “Edge Plane”.

The individual IoT workload can comprise of (n) number of accelerators. These accelerators can perform a unique function such as provisioning a device, control & commanding a device, patching device, provisioning AWS IoT Greengrass Core etc. To learn more about function or use-case specific accelerator refer to the AWS Connected Device Framework for more information. This framework can serve as the foundational building block for this architecture.

Isolating Accounts using AWS Organizations

Figure 4. Account and Access management using AWS Organizations and Organizational Units

We now extend the guidance through the use of AWS specific services. AWS Organizations in this case allows customers to use Organizational Units (OUs) that provide capabilities to the accounts within those OUs. All OUs apply their own guardrails for the accounts as well as governance for the tenant accounts. We utilize three different OUs in Figure 4.

1/ Shared Services Organizational Unit

The Single Pane of Glass resides within the Shared Services OU. It has its own account which hosts the aggregated dashboard. The OU in this case provides the capabilities to the own platform and grants access to the different user types to access the data they are allowed to see.

2/ Workloads Organizational Unit

The workloads OU hosts has multiple accounts, one account per tenant. It permits the users coming from the Single Pane of Glass access to their workloads and the outbound and inbound data from IoT Devices.

3/ Suspended Organizational Unit

Workloads in the suspended OU are no longer active but still part of the setup within AWS which allows for later investigation as well as deletion once no longer needed. That suspension can occur automatically base on criteria defined by the system administrators.

Event Driven Solution

Figure 5. Amazon EventBridge and AWS IoT Core integration.

In section we add the use of Amazon EventBridge integrated with AWS IoT Core. That approach allows for an event driven solution which will interact with the control pane or Single Pane of Glass. The control plane account will have Amazon EventBridge set up to send and receive messages to and from the individual workload OU accounts. This integration allows to invoke the different IoT workloads in the accounts and also the collection of data from the individual devices up to a aggregated view in the control plane account. Cross account interactions require specific permission which can be understood in more detail in the Service control policies (SCPs) documentation.

The workload OU accounts subscribe to the messages coming from the control plane side, and vice versa. Each workload is isolated by an individual tenant account, which also allows for cost isolation and thus achieves the tenancy model we needed.

Single Pane of Glass Architecture

Figure 6. Final Architecture for monitoring and managing a multi-region multi-account IoT workload

Finally we can focus on a diagram (Figure 6) with all the elements of the Single Pane of Glass architecture.

Starting from the right side we have multiple devices connected to AWS IoT Core. First, we focus our attention to the connection from AWS IoT Core into the workload. As the workload interacts with IoT devices through AWS IoT Core, Amazon EventBridge can be configured to react to specific events. Those events will then be passed onto the Single Pane of Glass accounts, where the user has access only to the relevant data and alarms.

Now we turn our attention to the connection from AWS IoT Core on the right side to AWS IoT Device Defender. Natively integrated with AWS IoT Core, AWS IoT Device Defender will execute auditing and monitoring tasks, reporting any anomalies or non-compliance to the reporting pipeline. Also, AWS IoT Device Defender natively integrates with AWS Security Hub and can be configured to deliver the audit finding to this service (read more about the native integration here). Respectively, AWS Security Hub is integrated cross account, delivering the alarms to IT administrators and delegating actions to operations if necessary.

This architecture allows the security operations team as well as IoT platform admins access to security insights and findings across the different accounts and regions.

Few examples of deviations that should be shared with security operation teams using AWS Security Hub are:

MQTT-based data exfiltration: Data exfiltration occurs when a malicious actor carries out an unauthorized data transfer from an IoT deployment or from a device. The attacker launches this type of attacks through MQTT against cloud-side data sources.
Impersonation: An impersonation attack is where attackers pose as known or trusted entities in an effort to access AWS IoT cloud-side services, applications, data, or engage in command and control of IoT devices.
Command and control, malware and ransomware: Malware or ransomware restricts your control over your devices, and limits your device functionality. In the case of a ransomware attack, data access would be lost due to encryption the ransomware uses.

If you want to find out more about the different security use cases covered by AWS IoT Device Defender you can access here.

Conclusion

In this blog post we walked you through the considerations for building a Single Pane of Glass for your multi-tenant IoT workloads when considering enterprise-wide security operations. With this approach now your IT teams and OT teams can rely on a single place for cyber-security posture, as well facilitate the standardization of already existing best practices and organization requirements.

For further learning and reading about AWS IoT solutions and ways to improve the overall security of your environment, please visit the following blog posts:

If you want to know more about designing and building a multi-tenant architecture for your AWS IoT environment, you can follow this workshop.

About the authors

Katja-Maja Kroedel is IoT Specialist Solution Architect at Amazon Web Services. She works with AWS customers to provide guidance on cloud adoption, migration, and strategy in the area of IoT. She is passionate about technology and enjoys building and experimenting in the cloud with innovative services, such as AWS IoT Device Defender. Katja has a Computer Engineering background and already worked at different roles within AWS, starting with her Masterthesis as well as her role as Generalist Solutions Architect in Germany, helping small- and middle-sized customers grow and learn about the cloud.

Leo Da Silva is a Security Specialist Solutions Architect at AWS and uses his knowledge to help customers better utilize cloud services and technologies securely. Over the years, he had the opportunity to work in large, complex environments, designing, architecting, and implementing highly scalable and secure solutions to global companies. He is passionate about football, BBQ, and Jiu Jitsu — the Brazilian version of them all.

Hassan Khokhar is a Sr. IoT Architect working in the Emerging Technologies, Engineering and Robotics practice part of Proserve. Hassan loves solving challenging problems for his customers by architecting & building frameworks and solutions to accelerate IoT implementations. Over they years he had opportunity to work for small and large companies helping them deliver IoT platforms and scale implementations.

The Internet of Things on AWS – Official Blog