Explore the Tokyo Olympics data journey! We ingested a CSV from GitHub into Azure using Data Factory, stored it in Data Lake Storage Gen2, performed transformations in Databricks, conducted advanced analytics in Azure Synapse, and visualized insights using Synapse or Power BI.

Azure projects - End to End Data Engineering Project with medallion architecture using Azure Data Factory & Azure Databricks. Azure Serverless/Logical DataWarehouse using Azure Synapse Analystics to demo CETAS, Data Modeling, Incremental loading, CDC and Sql Monitoring the data processing connected to Power BI

Azure Data Engineering project: Master data ingestion, transformation, and storage using Azure Data Factory, Databricks (PySpark), and Delta Lake. NYC Taxi data provides real-world context.

This project analyzes crime trends in Ireland, focusing on yearly shifts, top offense types, and crime distribution across counties. It highlights high-crime areas, with special attention to Dublin and nearby crime locations. The study identifies the most common crime, providing actionable insights for prevention strategies

End-to-End Azure Data Engineering Project demonstrates the implementation of a full-scale, scalable data engineering pipeline leveraging Azure cloud technologies. The project follows the Medallion Architecture to process and transform large volumes of data from multiple sources such as CSV, XLSX, JSON, PostgreSQL, SQL databases, and APIs.

End-to-end Flood Monitoring System (Azure ETL+ Power BI dashboard) using a flood API

This is an End to End Azure Data Engineering project copying data from Rest API to Azure cloud.

This repository contains code and resources for analyzing patents using Apache Spark, Python, and AWS services. The objective of this project is to extract insights and trends from patent data to inform business decisions and intellectual property strategies.

This Python-based project extracts data from Wikipedia using Apache Airflow, cleans it and pushes it Azure Data Lake for processing and further processing and visualization is done on Tableau.

This project demonstrates the end-to-end process of building a data pipeline using Azure Synapse Analytics, Azure Data Factory (ADF), Databricks, and Delta Lake to ingest, clean, transform, and store data.

Tokyo Olympics Analysis Using AZURE platform

Tokyo Olympic 2021 Analysis Using Microsoft Azure platform

This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Artificial Intelligence topics.

Azure data migration project to migrate data from on-prem SQL Server to Azure cloud using meta-data driven approach.

This repository contains a full setup for a Snowflake and Azure Data Factory pipeline, including a detailed README, a sample sales dataset for data ingestion and transformation, and Snowflake-generated analysis charts. It's ideal for learning Snowflake-Azure integration and automating data workflows.

This project creates an end-to-end data pipeline and interactive dashboard for analyzing mutual funds' performance using Microsoft Azure and Power BI. It leverages Azure Data Factory, Data Lake Storage, SQL Database, and Databricks to build a scalable, efficient pipeline, providing real-time insights and data-driven decision-making.

This repository contains resources for mastering Azure Data Factory from scratch, specifically designed for data engineers. The repository includes pipelines, datasets, linked services, data flows, and an input file that can be used as a practical guide to understand and implement various data engineering tasks in Azure Data Factory.

END TO END DATA ENGINEERING PROJECT