Amazon OpenSearch Ingestion now supports AWS Lambda for custom data transformation
Amazon OpenSearch Ingestion now allows you to leverage AWS Lambda for event processing and routing, enabling complex transformation and enrichment of your streaming data. Customers can now define custom Lambda functions in their OpenSearch Ingestion pipelines for use cases like generating vector embedding and lookups in external databases to power advanced search use cases.
OpenSearch Ingestion gives you the option of either using only Lambda functions or chaining Lambda functions with native Data Prepper processors when transforming data. You can also batch events into a single payload based on event count and size before invoking Lambda to optimize the number of Lambda invocations to reduce costs and improve throughput. Furthermore, you can use this feature with the inbuilt conditional expressions in Amazon OpenSearch Ingestion to enable use cases like sending out emails and notifications for real-time alerting.
This feature is available in all the 15 AWS commercial regions where Amazon OpenSearch Ingestion is currently available: US East (Ohio), US East (N. Virginia), US West (Oregon), US West (N. California), Europe (Ireland), Europe (London), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Singapore), Asia Pacific (Mumbai), Asia Pacific (Seoul), Canada (Central), South America (Sao Paulo), and Europe (Stockholm).
To learn more, see the Amazon OpenSearch Ingestion webpage and the Amazon OpenSearch Service Developer Guide.