Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

Construct Apache Airflow DAGs Declaratively via YAML configuration files

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects

A Python 3 package for learning Bayesian Networks (DAGs) from data. Official implementation of the paper "DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization"

This provider contains operators, decorators and triggers to send a ray job from an airflow task

Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows

Stop creating bad DAGs! Use this tool to measure and compare the parse time of your DAGs, identify bottlenecks, and optimize your Airflow environment for better performance.

This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow orchestration tool on AWS EC2 instance.

Opinionated framework based on Airflow 2.0 for building pipelines to ingest data into a BigQuery data warehouse

dagrad is a Python package that provides an extensible, modular platform for developing and experimenting with differentiable (gradient-based) structure learning methods.

Projeto final de conclusão do Bootcamp IGTI 2021/1

This a repo that was created to learn more about Airflow and develop awesome data engineering projects. 🚀🚀

The dag-checks consist of checks that can help you in maintaining your Apache Airflow instance.

ETL e Transformações

A complete NBA analytics data pipeline using PostgreSQL as the data warehouse, dbt for transformations, Apache Superset for visualization, all orchestrated with Apache Airflow deployed on Astronomer Astro.

This is a repo to keep track of all my projects related to data engineering

Setup Airflow and Implement Some Operators Deployed on K8s

This repo consist of codes to fetch covid data and store them into bigquery, scheduled by apache-airflow.

Projeto para Extração de dados de um SQL Server e Carga em um Data Lake Storage Gen2 da Azure (ADLS) usando Airflow Astro CLI e spark para movimentação dos dados entre as camadas tipo medalhão com delta lake.