How to detect and monitor Amazon Simple Storage Service (S3) access with AWS CloudTrail and Amazon CloudWatch

While protection of data is critical, equally important is observing who accesses it. AWS services allow you to control your data by determining where it’s stored, who has access, and how it’s secured.

AWS CloudTrail provides an effective way to track data access activities. You can detect access attempts, and identify potential unauthorized attempts. CloudTrail, captures different types of events related to your AWS resources, including management and data events. CloudTrail management events focus on capturing control plane operations to show management operations that are performed on AWS resources. Supported resources for data event types include Amazon S3 buckets, Amazon DynamoDB tables, and Amazon Relational Database Service (Amazon RDS). To see a full list of supported resources, visit logging data events.

In this blog post, we will explore how to use CloudTrail Data Events to monitor a sensitive file within a S3 bucket. We will set up an alarm with Amazon CloudWatch, and receive notifications via Amazon Simple Notification Service (Amazon SNS) when sensitive data access attempts are detected.

Prerequisites

1. An S3 Bucket with folders and objects

2. Configure CloudWatch Logs to monitor CloudTrail logs

Architecture overview

In this use case, consider a company with hundreds of S3 buckets and objects that needs to monitor access to sensitive objects at the object level. Rather than at the bucket or prefix level. To achieve this, it will require visibility into your access patterns. To achieve this level of monitoring, companies can leverage AWS CloudTrail data events to track and record specific S3 object-level activities.

CloudTrail integrate seamlessly with CloudWatch, enabling businesses to build custom monitoring solutions. These solutions can be tailored to track S3 activities in detail and set up automated alerts based on specific events or patterns. By implementing this granular monitoring approach using CloudTrail and CloudWatch, companies can enhance their control and security over sensitive data. Particularly in large-scale S3 environments with complex access requirements.

[Figure 1 – Architecture overview]

The objective of the use cases:

CloudTrail data events for Amazon S3 objects are enabled and are being delivered in a CloudWatch Log Group.
Data access attempts are monitored through metric filters to detect log entries in the log data and CloudWatch monitors access to the sensitive files.
When the sensitive files are accessed, CloudWatch triggers an alarm which is configured to send notifications to an SNS Topic.
Amazon SNS in return sends an email notification to alert administrators about the detected event.

Walkthrough

You will need an S3 bucket with information you want to try accessing. For this scenario, we will use an S3 bucket with an object that we will classify as sensitive.

[Figure 2 – AWS Console – S3 Bucket]

Set Up CloudWatch Metric and Alarms

You can use Amazon CloudWatch Logs to store, query, and monitor your log files from CloudTrail.

Depending on the naming convention of your CloudTrail log group, select the CloudTrail log group within CloudWatch.

[Figure 3 – AWS Console – CloudWatch Logs]

2. Select Metric Filters, and select Create a Metric Filter to monitor access attempts to the sensitive object. Make sure you replace the object ARN in the following Metric Filter Pattern:

Metric Filter Pattern:

$.eventName = "GetObject" && $.resources[0].ARN = "arn:aws:s3:::aws-cloudtrail-data-events-demo/sensitive/secrets.docx" }

Metric Namespace: CloudTrail-data-events
Metric Name: CloudTrailDataEventFileAccessAttemptMetrics

[Figure 4 – AWS Console – CloudWatch Metric Filters]

3. Review the created metric filter and select Create.

[Figure 5 – AWS Console – CloudWatch Metric Filters]

In other cases, customers want to monitor access of those objects for attempts that don’t have permissions to view those objects. They will also want visibility into access denied requests for those objects as well. In this case, you can use the preceding metric filter and add a condition to the statement:

$.eventName = "GetObject" && $.resources[0].ARN = "arn:aws:s3:::aws-cloudtrail-data-events-demo/sensitive/secrets.docx" && $.errorCode = “AccessDenied”}

4. Create a CloudWatch Alarm for accessing the file:

Alarm Name: Sensitive_File_Access
Trigger Condition: The threshold is set to one or more access attempts within one minute. You can change this based on how many attempts are acceptable before getting notified.
Alarm Actions: Send a notification to an SNS topic when the alarm state changes.

[Figure 6 – AWS Console – CloudWatch Alarm]

5. Select Next, and for Notification, select an existing or create a new SNS topic to notify the appropriate people when the alarm is invoked.

[Figure 6 SNS for CloudWatch Alarm]

6. Review the alarm and select Create.

The alarm will go into alarm state when the object is accessed. For the purpose of this scenario, this will occur when we try attempt to access the object.

Accessing the Amazon S3 file

For the purpose of this demo, you can try accessing you file through the AWS CLI, Console, or via AWS CloudShell. You will need the Amazon S3 URI for the sensitive object to run the command:

aws s3 cp s3://[foldername/filename]

Alternately, you can access your object using AWS SDK.

From the previous section, you set the Amazon S3 Bucket to capture CloudTrail Data events. When you tried to access this Amazon S3 Bucket, it created an event and captured it in the S3 bucket.

In the email notification, you will see information of the alarm and when it was invoked. If you want to integrate with Security information and event management (SIEM) tools for Event Notifications, visit Moving to the cloud? Here is how you can integrate SIEM on AWS.

[Figure 8 – Email notification]

Navigate to CloudWatch and select the alarm that you created previously, you can see the alarm state “In alarm.”

[Figure 9 – CloudWatch Alarms]

By sending CloudTrail logs to CloudWatch, you can monitor specific activity. And setting up CloudWatch alarms for specific events, you can quickly identify and mitigate such incidents. Additionally, this can help preventing unexpected cost escalation from the misuse of an S3 bucket.

By combining CloudTrail detailed event logging with CloudWatch monitoring and alerting capabilities, you gain better visibility and control over your S3 usage. This can help you understand your Amazon S3 activities further. And periodically review your storage patterns to move objects if they’re no longer being accessed.

Conclusion

By establishing CloudTrail Data Events with CloudWatch, you can receive near real-time monitoring, and automated alerting for access to your Amazon S3 Buckets. This helps you to respond to potential data access events. And safeguard your organization’s valuable data assets. To explore custom metrics created from CloudTrail logs further, you can learn more about “Simplify analysis of AWS CloudTrail data leveraging Amazon CloudWatch machine learning and advanced capabilities.”

AWS Cloud Operations Blog