How you can use Amazon GuardDuty to detect suspicious activity within your AWS account

September 9, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

Amazon GuardDuty is an automated threat detection service that continuously monitors for suspicious activity and unauthorized behavior to protect your AWS accounts, workloads, and data stored in Amazon S3. In this post, I’ll share how you can use GuardDuty with its newly enhanced highly-customized machine learning model to better protect your AWS environment from potential threats. The model works at scale and across a broad spectrum of customer use cases, workloads, and operating models. The new findings available to you in GuardDuty allow you to use anomaly detection to generate security relevant results that are differentiated from unusual but benign, without a lot of false alarms or noise.

Overview of the new GuardDuty machine learning model

The new GuardDuty machine learning model operates on the continuous stream of API invocations that occur in your AWS accounts, based on user activity that is tracked in AWS CloudTrail. The model is based on Variational Autoencoders, which have recently emerged as one of the most successful approaches to model complex data distributions. This allows GuardDuty to learn the probability distribution of API calls invoked by users in your AWS account. Even if a user in your account operates in a certain way for the first time, the model will predict whether it is part of their normal, expected operations. It’s this added probabilistic nature that helps to produce accurate detections of suspicious behavior within your account.

GuardDuty now uses this new machine learning model to identify unusual activity within your accounts, analyze the security relevance of the activity, given the context in which it was invoked, and apply predictive probability to make a final verdict on whether that activity is sufficiently anomalous to warrant investigation. The new model-based threat detections added to GuardDuty will also help you identify the attack tactic associated with the anomalous API invocations, including discovery, initial access, persistence, privilege escalation, defense evasion, credential access, impact, and data exfiltration.

The new GuardDuty findings have shown an average decrease in anomalous account activity alerts by over 50%, while also expanding the monitoring coverage by over 300%. The anomalous user behavior these new findings flag is more likely to represent actual malicious activity compared to the previous anomaly detections in GuardDuty.

Suspicious behavior the machine learning model can now help you detect includes:

• Unusual Amazon Simple Storage Service (Amazon S3) discovery API calls along with various levels of access permission modifications.
• Unauthorized creation of new Amazon Identity and Access Management (IAM) users, roles, or access keys to persist access.
• Resource hijacking of Amazon Elastic Compute Cloud (Amazon EC2) instances for use in cryptocurrency mining.
• Generating Amazon Relational Database Service (Amazon RDS) snapshots as part of data exfiltration attempts.

If you are an existing GuardDuty customer, then the new machine learning threat detections are already available to you as part of the service. As with all GuardDuty detections, we work to continually expand and improve them, and new detections are enabled by default for all your GuardDuty-enabled AWS accounts, at no additional cost. This means that as you are reading this, the model is tracking API activity by users in your accounts across dozens of AWS services, including Amazon EC2, Amazon Identity and Access Management (IAM), Amazon Simple Storage Service (S3), Amazon DynamoDB, Amazon Elasticsearch Service, Amazon Relational Database Service (RDS), AWS Key Management Service (KMS), Amazon Elastic Container Registry (Amazon ECR), and AWS Secrets Manager.

Enriched metadata in the alerts to help you quickly triage new threat detection

The new machine learning behavioral detections in GuardDuty give you rich contextual data as part of the GuardDuty findings. The new GuardDuty findings allow you to make quick, on-the-spot decisions about whether to trigger an incident response workflow. You can view this enriched context in the GuardDuty console, and the full detail is included in the finding JSON pushed out through Amazon EventBridge and published to AWS Security Hub.

In the new findings, you can now see a detailed list of all anomalous APIs that were invoked by the same user within a window of several minutes. For example, instead of telling you that IAM user Admin-1 anomalously invoked the S3 ListBuckets API, GuardDuty will tell you that in proximity to that operation the user also anomalously invoked the GetBucketACL, GetBucketPublicAccessBlock, and PutBucketPublicAccessBlock APIs. All the APIs on the list will be grouped by their associated AWS service to make it even easier for you to quickly understand which services were accessed as part of the anomalous activity. The finding will also provide details on which of the anomalously invoked APIs were called successfully and which ones failed, including the error response received. This context can be important as you triage the findings. For example, successful API invocations indicate the user was able to perform the operations and could represent a higher severity incident. On the other hand, if the user’s access was denied, this could indicate compromised credentials with an attacker attempting to identify what access permissions are available.

Each finding will explicitly call out which attributes of the activity were unusual for the specific user, as well as for all other users that operate in the same AWS account to help you identify why the behavior was flagged as highly suspicious. Finally, the finding details also include information on what the expected behavior is for the user, and for all other users that operate in the same AWS account. The expected behavior will be grouped based on its frequency during the profiling period. For example, frequent (daily or weekly), infrequent (a few times a month), or rare (less than once a month).

How to use the new findings in the GuardDuty console

Let’s walk through an example. As you triage alerts in the GuardDuty console, you see an alert on anomalous discovery related activity shown in Figure 1:

Figure 1: Discovery:IAMUser/AnomalousBehavior finding Anomalous APIs section view

Under the Anomalous APIs section, you see that the user successfully invoked three DynamoDB and RDS related APIs that are associated with discovery tactics: ListTables, DescribeTables, and DescribeDBSnapshots.

Working your way down the findings detail pane, next you see the Unusual behavior sections shown in Figure 2.

Figure 2: Discovery:IAMUser/AnomalousBehavior finding unusual behavior section view

Analyzing this section, you learn that the activity was conducted from a remote IP associated with a network that is unusual for this user, as well as for all other users that operate in the same AWS account. You can also see that it is unusual for this user to invoke the APIs that were used to list and describe DynamoDB tables and to describe RDS database snapshots. Below the Unusual behavior sections you see three additional sections that provide additional important details on the activity that are shown in Figure 3.

Figure 3: Discovery:IAMUser/AnomalousBehavior finding additional sections

The Resource affected section helps you answer important questions about the AWS IAM user associated with the activity, including the user name and user type. The Action section allows you to dive deeper on one of the API actions that was part of the activity, including the user agent that was used as part of the activity. Finally, under the Actor section you can see information on the remote IP used, including the geolocation and associated network.

At this point, this activity already looks highly suspicious, as the actor operated from a network that no other user in your AWS account has operated from, and the APIs invoked are not part of profiled operations. However, there is more you can review before making a final verdict. Diving one layer deeper, you can review what behavior is expected for the user, as well as for the rest of the users in this AWS account. Returning to the Anomalous APIs section (shown in Figure 1), you can choose the Usual APIs button to open up the dialog box shown in Figure 4.

Figure 4: Discovery:IAMUser/AnomalousBehavior usual APIs dialog box

The Usual APIs dialog box shows you a list of the top-20 APIs that are most commonly invoked both by the user (under User Identity on the left hand side), as well as all other users that operate in your AWS account (under All users in account on the right hand side). You can quickly scan the list and see that invoking APIs associated with DynamoDB and RDS is not part of common operations in your AWS account. Rather, this specific user often invokes APIs associated with Amazon GuardDuty, while the rest of the users in your account often invoke APIs associated with S3 buckets and EC2 instances.

To finish the triage process, you check the usual behavior for both the AWS account and the user that invoked the anomalous APIs. You return to the Unusual behavior (Account) section (shown in Figure 2) and choose the Usual behavior button to open up the dialog box shown in Figure 5.

Figure 5: Discovery:IAMUser/AnomalousBehavior Account usual behavior

Then, you go to the Unusual behavior (User Identity) section (shown in Figure 2) and choose the Usual behavior button to open up the dialog box shown in Figure 6.

Figure 6: Discovery:IAMUser/AnomalousBehavior User Identity usual behavior

The Usual behavior (Account) dialog box in Figure 5 shows the expected behavior across all users in your account. The Usual behavior (User Identity) dialog box in Figure 6 shows the expected behavior for the user that invoked the anomalous APIs. There are a number of different tabs in each dialog box that provide you with information about various attributes of the expected behavior. Focusing on the commonly used networks, you can see that users in this AWS account, including the specific user that invoked the anomalous APIs, usually operate from the Amazon network, or from networks of common US-based service providers. This provides you with further evidence that the anomalous behavior detected by GuardDuty is highly suspicious.

Start an investigation with Amazon Detective

After you identify that the activity detected is clearly suspicious, you can choose the Investigate with Detective link at the top of the finding detail pane shown in Figure 7, to start an Amazon Detective investigation.

Figure 7: Discovery:IAMUser/AnomalousBehavior launching an Amazon Detective investigation

Amazon Detective complements Amazon GuardDuty by collecting log and event data from sources such as AWS CloudTrail and Amazon Virtual Private Cloud (VPC) Flow Logs. Detective organized the data into an analytics-driven graph model that summarizes resources, user behaviors, and associated interactions observed across all enabled accounts for up to the last 12 months. Detective uses this data to produce tailored visualizations that summarize network and user activity across all your enabled accounts.

Detective can help you quickly answer questions such as the following:

• Did this user do any forward role assumptions (role chaining) to conduct other activity under a different role after assuming the admin role?
• What other roles did this user assume across my accounts around the time of this finding, and what APIs did they invoke?
• When was activity from the remote IP address first observed across my accounts?
• What users and roles have accessed my AWS resources from this IP address, and what API calls have they invoked? Which of these calls failed, and which ones succeeded?
• Has this IP address sent or received any data from any of my EC2 instances? If so, how much, for how long, and on which ports?

By providing visual analytics and summaries collected from management events and network traffic, Detective helps you identify the root cause of security issues. Being able to quickly see patterns of activity while being able to drilldown and understand the details, help you quickly understand how long issues have been going on for and what needs to be remediated.

Summary of new and deprecated findings

Here is a summary of all the newly added finding types.

New GuardDuty finding types

1. Discovery:IAMUser/AnomalousBehavior
   Severity: Low
   Description: This finding informs you that an anomalous API request was observed in your account. The API observed is associated with the discovery stage of an attack, in which an unauthorized user gathers information to determine if your AWS environment is susceptible to a broader attack. APIs in this category are typically get, describe, or list operations, such as, DescribeInstances, GetRolePolicy, or ListAccessKeys.
2. InitialAccess:IAMUser/AnomalousBehavior
   Severity: Medium
   This finding informs you that an anomalous API request was observed in your account. The API observed is commonly associated with the initial access stage of an attack, in which an unauthorized user attempts to establish access to your environment. APIs in this category are typically get token, or session operations, such as, GetFederationToken, StartSession, or GetAuthorizationToken.
3. Persistence:IAMUser/AnomalousBehavior
   Severity: Medium
   This finding informs you that an anomalous API request was observed in your account. The API observed is commonly associated with persistence tactics, in which an unauthorized user has gained access to your environment and attempts to maintain that access. APIs in this category are typically create, import, or modify operations, such as, CreateAccessKey, ImportKeyPair, or ModifyInstanceAttribute.
4. PrivilegeEscalation:IAMUser/AnomalousBehavior
   Severity: Medium
   This finding informs you that an anomalous API request was observed in your account. The API pattern is commonly associated with privilege escalation tactics, in which an unauthorized user attempts to gain higher-level permissions to an environment. APIs in this category typically involve operations that change IAM policies, roles, and users, for example: AssociateIamInstanceProfile, AddUserToGroup, PutUserPolicy.
5. DefenseEvasion:IAMUser/AnomalousBehavior
   Severity: Medium
   This finding informs you that an anomalous API request was observed in your account. The API observed is associated with defense evasion tactics, in which an unauthorized user is attempting to cover their tracks and evade detection. APIs in this category are typically delete, disable, or stop operations, for example: DeleteFlowLogs, DisableAlarmActions, StopLogging.
6. CredentialAccess:IAMUser/AnomalousBehavior
   Severity: Medium
   This finding informs you that an anomalous API request was observed in your account. The API request observed is associated with the credential access stage of an attack, in which an unauthorized user attempts to collect passwords, usernames, and access keys for your environment. The APIs in this category include GetPasswordData, GetSecretValue, and GenerateDbAuthToken.
7. Impact:IAMUser/AnomalousBehavior
   Severity: High
   This finding informs you that an anomalous API request was observed in your account. The API observed is commonly associated with impact tactics, in which an unauthorized user attempts to disrupt operations and manipulate, interrupt, or destroy data in your account. APIs in this category are typically delete, update, or put operations, for example: DeleteSecurityGroup, UpdateUser, PutBucketPolicy.
8. Exfiltration:IAMUser/AnomalousBehavior
   Severity: High
   This finding informs you that an anomalous API request was observed in your account. The API observed is commonly associated with exfiltration tactics, in which an unauthorized user attempts to exfiltrate data from your AWS environment. APIs in this category are typically related to S3, database services, and EC2, for example: PutBucketReplication, CreateSnapshot, RestoreDBInstanceFromDBSnapshot.

These newly added finding types replace some of the previous finding types, which are now deprecated.

Deprecated GuardDuty finding types

The following GuardDuty finding types are deprecated. This deprecation took effect on March 12, 2021. If any of the previously available finding types were generated in you AWS environment prior to March 12, 2021, then they will be deleted within 90-days of the time they were generated, in accordance with finding retention period in GuardDuty. However, no future version of these findings will be generated for your AWS environment.

1. Persistence:IAMUser/NetworkPermissions
2. Persistence:IAMUser/ResourcePermissions
3. Persistence:IAMUser/UserPermissions
4. Recon:IAMUser/NetworkPermissions
5. Recon:IAMUser/ResourcePermissions
6. Recon:IAMUser/UserPermissions
7. ResourceConsumption:IAMUser/ComputeResources
8. Stealth:IAMUser/LoggingConfigurationModified
9. UnauthorizedAccess:IAMUser/ConsoleLogin
10. Discovery:S3/BucketEnumeration.Unusual
11. Impact:S3/PermissionsModification.Unusual
12. Impact:S3/ObjectDelete.Unusual
13. PrivilegeEscalation:IAMUser/AdministrativePermissions

For more information about the new detections, as well as the ones that have been deprecated, see Finding types in the Amazon GuardDuty User Guide.

Conclusion

If you are already using Amazon GuardDuty today then no further action is required in order for you to start using this new capability to view the findings in the console. If you have set up GuardDuty to push findings through AWS EventBridge for downstream consumption by workflow tools, then you should verify that your EventBridge rules are configured to deliver these newly added findings.

At AWS, we are committed to continually improving GuardDuty, to make it even easier for you to operate securely in AWS. Keep the feedback coming as it’s what powers us at AWS. If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the Amazon GuardDuty forum or contact AWS Support.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.