/projects
Snowflake ELT Pipeline with dbt and Airflow
End-to-end ELT pipeline extracting TPCH orders into Snowflake and transforming with dbt, orchestrated by Apache Airflow.
Highlights
- Built an automated workflow using Airflow to orchestrate dbt transformations on Snowflake (daily schedule, no catchup)
- Defined dbt project structure with staging and mart models, macros for utility functions, and tests for data quality
- Integrated Snowflake via a ProfileConfig mapping and used a virtualenv-executable dbt run inside the Airflow DAG
Technologies
SnowflakedbtApache AirflowPythonSQLELT Pipelines