Amazon Data Firehose supports continuous replication of database changes to Apache Iceberg Tables in Amazon S3
Amazon Data Firehose now enables capture and replication of database changes to Apache Iceberg Tables in Amazon S3 (Preview) . This new feature allows customers to easily stream real-time data from MySQL and PostgreSQL databases directly into Apache Iceberg Tables.
Firehose is a fully managed, serverless streaming service that enables customers to capture, transform, and deliver data streams into Amazon S3, Amazon Redshift, OpenSearch, Splunk, Snowflake, and other destinations for analytics. With this functionality, Firehose performs an initial complete data copy from selected database tables, then continuously streams Change Data Capture (CDC) updates to reflect inserts, updates, and deletions in the Apache Iceberg Tables .This streamlined solution eliminates complex data pipeline setups while minimizing impact on database transaction performance .
Key capabilities include: • Automatic creation of Apache Iceberg Tables matching source database schemas • Automatic schema evolution in response to source changes • Selective replication of specific databases, tables, and columns
This preview feature is available in all AWS regions except China, AWS GovCloud (US), and Asia Pacific (Malaysia) Regions. For terms and conditions, see Beta Service Participation in AWS Service Terms.
To get started, visit Amazon Data Firehose documentation and console.
To learn more about this feature, visit this AWS blog post.