Arcane Insight is a data analytics project designed to harness the power of SQLMesh & DuckDB to collect, transform, and analyze data from Blizzard’s Hearthstone API. Focused on card statistics and attributes, this project reveals detailed insights into card mechanics, strengths, and trends to support BI and strategic analysis.

This project implements a Lakehouse Medallion Architecture using modern Data Stack tools such as Fivetran, Snowflake and dbt. The ficticious organization is an e-commerce company.

'Talk to Your Factory' leverages Industrial IoT, Edge & Cloud Computing, and Generative AI to streamline factory operations. It allows real-time, natural language communication with factory systems, helping operators quickly identify issues, boost efficiency, and minimize downtime.

Extract data from many databases of Labor, Invalids and Social Affairs sectors and convert to appropriate structure and format, then upload to shared data warehouse and data mart. Thanks to that, people of state agencies can easily retrieve and analyze data based on the compiled data warehouse.

Implemented enhanced analytics for trip transactions and ride-based data from SQL Server using Medallion Architecture with Azure Data Factory and Databricks.

This repository contains a comprehensive end-to-end Azure Data Engineering solution, covering the entire data lifecycle from ingestion to reporting. The project utilizes various Azure services and tools to achieve efficient data handling, transformation, and reporting.

Tracking Tweet sentiment at scale using a pretrained Huggingface transformer (classifier) Model

In addition to exploring good practices in Python, the purpose is to apply concepts from Medallion Architecture, object-oriented programming and ETL, using public social data from Brazilian municipalities. This repository also uses DuckDB and Streamlit to display results

This repository demonstrates the creation of a robust, 3-tier data lake using AWS resources.

Scalable DBT project with Medallion Architecture & SQL Server. Unlocking insights for smarter urban planning.

In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider.

Developed an end-to-end real time sentiment analysis system using Delta Lake Medallion architecture, with Hugging Face Transformer Model, Spark, and MLflow integration.

This project demonstrates the end-to-end process of building a data pipeline using Azure Synapse Analytics, Azure Data Factory (ADF), Databricks, and Delta Lake to ingest, clean, transform, and store data.

Building a Data Lakehouse using the Medallion architecture.

Data Warehouse with ELT pipelines