Category: Application Integration

Apache Airflow‘s active open source community, familiar Python development as directed acyclic graph (DAG) workflows, and extensive library of pre-built integrations have helped it become a leading tool for data scientists and engineers for creating data pipelines. Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a fully managed service that makes running open source […]

Amazon is excited to open source a key technology that powers EventBridge as Event Ruler under the Apache 2.0 license. Event Ruler is a Java library that allows you to build applications that can match any number of rules against events at several hundred thousand events per second.

GraphQL is an application-level query language that helps clients and servers communicate by establishing a common protocol for queries. It represents an alternative to the REST style: unlike REST, GraphQL gives the client, not the server, the power to define what kind of data will be included in the response to its query. GraphQL allows […]

This post was written by Dzenan Softic and Sam Dengler. Many organizations rely on Apache Airflow, an open source project, to orchestrate their data pipelines. In 2020, Amazon Web Services (AWS) released Amazon Managed Workflows for Apache Airflow (Amazon MWAA), which lets engineers focus on business solutions rather than on running and maintaining infrastructure for […]

Modern data science environments often involve many independent projects, each spanning multiple accounts. In order to maintain a global overview of the activities within the projects, a mechanism to collect data from the different accounts into a central one is crucial. In this post, we show how to leverage existing services—Amazon DynamoDB, AWS Lambda, Amazon […]

Many developers and data engineers use Managed Workflows for Apache Airflow (Amazon MWAA), a managed service for open source Apache Airflow, to programmatically author, schedule, and monitor workflows. With Amazon MWAA, you can focus on business logic and build workflows without worrying about the management responsibilities such as setup, patching, upgrades, security, scaling, and so […]

This post was written by Tomas Christ, Solution Architect at eprimo GmbH. eprimo GmbH is a wholly owned subsidiary of E.ON SE, situated near Frankfurt, Germany. It represents the largest purely green-energy supplier in Germany with some 1.7 million customers. Currently, eprimo has a staff of approximately 160 people. We have been using Amazon Web […]

Data scientists and engineers have made Apache Airflow a leading open source tool to create data pipelines due to its active open source community, familiar Python development as directed acyclic graph (DAG) workflows, and extensive library of prebuilt integrations. However, managing the connections and variables that these pipelines depend on can be a challenge, especially […]

Amazon Managed Workflows for Apache Airflow (MWAA) is not affected by the recently announced vulnerability in Apache Airflow 1.10.12. The default airflow.cfg file uses a temporary key that is the same for all installations. In Airflow 1.10.12 and earlier, there was no restriction in using that temporary key on the Airflow web server, meaning that […]

This post was contributed by AWS Serverless Hero Ben Ellerby. One of the most exciting things about serverless is the acceleration it can provide to teams. No longer do you have to invest time in the undifferentiated, heavy lifting of managing the infrastructure of our applications. Instead you can spend time writing code that creates […]