Learn how to use Apache Airflow to create dynamic, extensible, elegant, scalable data workflows on Google Cloud
Top Caserta consultants share, in technical detail, how to define and configure Airflow pipelines to create workflows that are maintainable, versionable, testable, and collaborative.
In this 50-minute webinar, you will learn best practices to create Airflow directed acyclic graphs (DAGs) of tasks, enforcing relationships and dependencies. The talk covers constructing Airflow pipelines with Spark based ETL, a Google Storage Data Lake, and a Big Query Data Warehouse.
The stage is shared with Dallas S. Simpson, Director of Data Engineering at SoulCycle Inc. Dallas co-presents with Caserta to share his experience and lessons learned using Airflow on their internal GCP Data Analytics Platform project.
Data architects, data engineers and data scientists alike will enjoy the contents of this talk.
Watch full video:
Presentations by:
Joe Caserta
Joe CasertaPresident, Caserta
Dallas S. Simpson
Director of Data Engineering at SoulCycle
Dovy Paukstys
Solutions Architect, Caserta