Streaming order book data using TD Ameritrade API

ETL motor racing data project using Azure Databricks, Pyspark and Azure Date Lakes

Real-time data engineering pipeline using Kafka, Spark Structured Streaming, Medallion architecture, dbt, and Azure Data Lake powering a PostgreSQL warehouse and Power BI analytics.

Azure Data Lake Gen2 storage connectors for Data Culpa - monitor data quality automatically with Data Culpa Validator

End-to-end Data Engineering platform with Medallion Architecture (Bronze → Silver → Gold) | Azure Data Lake | Polars | FastAPI | Docker

End-to-end real-time data pipeline using Kafka, Spark, Delta Lake, DuckDB, and Power BI. Simulates clickstream analytics with batch + streaming workflows for modern data engineering.

A multinational food safety analytics platform that integrates recall data from six government agencies across the US, EU, and UK into a data warehouse.

A production-grade IPL Data Engineering pipeline using Medallion Architecture on Azure Databricks, featuring PySpark, Delta Lake Auto Loader, 'MERGE' logic, and Unity Catalog.

ETL pipeline tailored for Olympics data

End-to-end retail analytics platform built with Python, Azure Data Factory, ADLS Gen2, Azure SQL, and Power BI for automated data ingestion, warehousing, and business insights.

A repository to continue my education on Azure Data Services.

Cloud Based Sensoring Solution for flow telemetry. Shown here is an early stage sensoring prototype with azure based alerting and app deployment.

Real-time Azure data pipeline using Event Hubs, Databricks, and ADLS

This repository contains code for an end-to-end IoT data pipeline using Azure services. It ingests, processes, and stores IoT device data from AWS S3 to Azure Data Lake Storage and Azure SQL Database, leveraging Azure Data Factory and Azure Functions for seamless integration and automation.

A data engineering project that scrapes football stadium data from Wikipedia, processes it with Airflow and Python, stores it in Azure Data Lake, moves it via Data Factory, queries with Synapse, and visualizes insights using Tableau.