The Internet of Things on AWS – Official Blog

Use AWS IoT Device Defender to detect statistical anomalies and to visualize your device security metrics

AWS IoT Device Defender supports your efforts to secure your IoT device fleet. AWS IoT Device Defender Detect establishes a baseline behavior for IoT devices and then identifies devices that do not conform as anomalous. AWS IoT Device Defender Detect operates on security profiles, which are applied to the entire device fleet, or to a thing group. From a security standpoint, security profiles specify your IoT device’s expected behavior. We recommend that you group devices that are expected to behave similarly into a thing group. Next, you should configure behaviors and thresholds for AWS IoT Device Defender metrics generated by IoT devices and set up appropriate actions when a violation occurs.

The following new features help customers configure thresholds, analyze device behavior in production deployments, and investigate potential issues:

Defender metrics visualization in the AWS management console makes it possible for you to view metrics generated by devices that belong to a security profile as statistical aggregations.

Configurable alerting means you can configure, per device, the number of consecutive violations after which an alert is triggered.

Statistical anomaly detection lets you express thresholds in terms of percentiles calculated across metrics generated by all devices in the security profile. You can also express thresholds in terms of static values.

In this blog post, I will show you how to set up AWS IoT Device Defender Detect using a scenario of 10,000 ID card scanners installed at all ingress/egress points of factories located in multiple cities. I will also show how these new features in AWS IoT Device Defender Detect help me in the process.

Let’s review the process of setting up in this use case. These ID card scanners have the following characteristics:

  • Every time the device scans an ID card, it captures some data about the card.
  • These devices collect the ID card data over an hour and upload it to the server.
  • These devices communicate to the server through TCP connections.

Threshold configuration

In this scenario, my objective is monitor the security of these ID card scanners and receive an alert if a security incident takes place.

Let’s start by breaking down a security incident. A malicious attacker (for example, someone behind a DDoS botnet or spam email campaign) must establish a communication channel to the IoT device, on which the malware and commands would be sent to the device.

The following are some metrics that can be captured that can help identify this security incident:

  • Remote Address: An IP address communicating with the device. In addition to legitimate IP addresses, this metric will also contain the attacker’s IP address.
  • Number of TCP connections: The number of connections between a server and the IoT device. In addition to legitimate communication between the server and the device, this metric will also contain the connection between the attacker and the device.
  • Incoming bytes: The number of bytes received by the device. In addition to legitimate communication with the device, this metric will also include the number of bytes that correspond to malware and commands sent by the attacker.
  • Outgoing bytes: The number of bytes sent from the device. In addition to the legitimate number of bytes sent by the device, this metric will also include the number of bytes sent to a server (DDoS) or user (spam email).

For the purpose of this demo, I will configure thresholds for the outgoing bytes metric, Bytes out.

I will begin by creating a security profile for these ID card scanners and collecting the metrics I’m interested in.

On the Expected behaviors page of the AWS IoT console, I enter a name and description for the security profile, and then choose Next.

Setting up of behaviors used to be a required part of creating a security profile. Now you can simply choose to retain the metrics you are interested in and configure the thresholds later.
For the purpose of this demo, I will not set up any alert target in this security profile:

Next, I will attach the security profile to the appropriate thing group (FactoryIDCardScanner_ThingGroup) and then choose Save.

When the security profile confirmation page is displayed, the process is complete.

Let’s assume that I collect a sufficient amount of metrics from these ID card scanners. Now let’s take a look at how the new features in AWS IoT Device Defender Detect help me configure appropriate thresholds for my security profile.

Defender metrics

The Defender metrics allow me to visualize metrics generated by all devices in a security profile as statistical aggregations in the AWS IoT management console. This visualization appears on the dashboard for my security profile. Let’s see the p0 (minimum value) and p100 (maximum value) values for the Bytes out metric. (Here “p” refers to percentiles.)

Note: Data is available for visualization as soon as it is received by AWS IoT Device Defender. Data is stored for a rolling window of 14 days. Customers don’t need to wait 14 days for visualizations to be populated.

By analyzing these graphs, I can make the following observations, which were unknown to me until now:

  • Bytes out varies between 0 to 17.5 kB.
  • There are peaks at 8 A.M. and at 8 P.M., which might simply indicate when people enter and leave the factory for the day.

Because there is variance in the metrics, it would be insufficient to say that Bytes out should be less than 17.5 kB (a static rule) because the baseline behavior for these devices is changing at each time interval. For situations like this, I would use statistical anomaly detection.

Statistical anomaly detection

To configure the threshold to Every device in this security group should behave in a certain percentile of all devices in this security group, on the Expected behaviors page, under Check Type, I choose Statistical Threshold. Under Statistical Threshold, I choose a percentile value.

The following shows the threshold in the security profile was successfully updated.

Configurable alerting

Anomalous behavior can be a result of a transient error, such as a temporary network outage. Alerts raised in response to such errors are false. To make anomaly detection more robust, I can configure AWS IoT Device Defender Detect to generate an alert only if a device has been in violation of the thresholds for a specified number of continuous time intervals.

It is important to emphasize that this configuration is a trade off between false and legitimate security alerts. For example, if the security incident were to last only for one time period, and if AWS IoT Device Defender Detect is configured to generate an alert only after two consecutive violations, the security incident will be missed and the appropriate alert action will not be taken.

To set up configurable alerting, on the Expected behaviors page, I would have to update values for the following:

  • Datapoints to alarm. (An alarm occurs if a device is in violation of a behavior for this many consecutive datapoints.)
  • Datapoints to clear. (An alarm is cleared if the device is no longer in violation of the behavior for this many consecutive datapoints.)

When an alarm occurs, I want to investigate the behavior of the device. The following view is available on the Defender metrics for that device. It provides me with access to the raw values.

Note: Metrics discussed in this blog post can only be observed from the device. To capture these metrics, customers must send them to AWS IoT Device Defender. (Sample implementation is provided here.)

Wrapping up

In this post, I’ve shown you how to set up a security profile, attach it to a thing group, observe statistical aggregations, and use statistical anomaly detection. I’ve also shown you how to tune the sensitivity of alerts using the configurable alerting feature.

Learn more