Session: Using Apache Airflow as a Container Orchestrator to Move Beyond Python
Apache Airflow is an open source orchestration tool that helps you to programmatically create workflows in Python that will help you run, schedule, monitor and mange data engineering pipelines in Python. But what if you want to use other tools or languages but leverage the benefits of Apache Airflow? Increasingly data engineering teams are containerising their applications, and then using Apache Airflow operators to orchestrate the execution. In this session, we will take a look at some of the options, patterns and different options you have to do this, using hands on demos and examples to show you.
