How to Ingest TBs of Data into Splunk with AWS Serverless Applications

By Tarik Makota, Solutions Architect at AWS

At AWS re:Invent 2017, we introduced the AWS Serverless Application Repository that enables Amazon Web Services (AWS) customers to discover, deploy, and publish serverless apps for data processing, stream processing, Internet of Things (IoT) device data telemetry, and more.

A serverless app follows the AWS Serverless Application Model (AWS SAM) format—it’s an AWS CloudFormation template that packages all the resources needed by a customer to deploy a serverless architecture.

Splunk is an AWS Partner Network (APN) Advanced Technology Partner with AWS Competencies in Big Data, DevOps, Security, Government, IoT, and Education. Its software and cloud services help customers deepen business insights, mitigate cybersecurity risk, prevent fraud, improve service, and reduce cost.

The Splunk AWS Serverless Applications are available from the AWS Lambda console and allow customers to ingest TBs of data into Splunk. As a Lambda blueprint launch partner in 2015, Splunk released two blueprints. As serverless adoption grew, they added more purpose-built blueprints based on customer requests for different data sources.

Learn more about Splunk AWS Serverless Applications >>

Splunk Lambda Blueprints for You to Use

In collaboration with AWS, Splunk ported six of its existing Splunk Lambda blueprints into Splunk Serverless Apps. These blueprints are available directly in the Serverless Application Repository:

1. splunk-logging: Log events from AWS Lambda itself to Splunk’s HTTP event collector
2. splunk-dynamodb-stream-processor: Stream Amazon DynamoDB events to Splunk’s HTTP event collector
3. splunk-elb-application-access-logs-processor: Stream Application Load Balancer access logs from Amazon Simple Storage Service (Amazon S3) to Splunk’s HTTP event collector
4. splunk-elb-classic-access-logs-processor: Stream Classic Load Balancer access logs from Amazon S3 to Splunk’s HTTP event collector
5. splunk-iot-processor: Stream AWS IoT Core events to Splunk’s HTTP event collector
6. splunk-kinesis-stream-processor: Stream events from Amazon Kinesis Video Stream to Splunk’s HTTP event collector

Sign up for a private preview to gain access to AWS Serverless Application Repository. When creating a new function in your AWS Lambda console, select “Serverless Application Repository” and search for “splunk” to see several purpose-built serverless apps.

Additional Resources

Check out the Splunk serverless apps that are open-sourced to dive deeper, including the underlying AWS SAM template. We welcome your contributions to help extend these apps into additional data sources and use cases that are important to you.

Our goal with Splunk AWS Serverless Apps is to further enable customers to leverage the flexibility, scalability, and cost-effectiveness of serverless computing. To help you trace, prioritize, and aggregate this data, explore the Data Insights Tool that was jointly created by AWS Marketplace and Splunk.

To analyze and deconstruct all this data from AWS services, see the various Splunk solutions available in AWS Marketplace, including Splunk Insights for AWS Cloud Monitoring.
.

Splunk – APN Partner Spotlight

Splunk is an AWS Competency Partner. Its software and cloud services enable customers to search, monitor, analyze, and visualize machine-generated big data from websites, applications, servers, networks, IoT, and mobile devices.

Contact Splunk | Solution Overview | Buy on Marketplace

*Already worked with Splunk? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.