Flood Monitoring Dashboard

This project is an end-to-end Flood Monitoring System (ETL+dashboard) built using Azure Blob Storage, Azure Data Factory, Azure Databricks, and Power BI. It integrates data from the Flood Monitoring API and provides actionable insights through an interactive Power BI dashboard.

---

Overview

Flooding can significantly impact infrastructure, communities, and ecosystems. This project aims to provide a real-time flood monitoring solution to:

• Track flood alerts.
• Analyze severity levels.
• Identify affected areas.
• Enable data-driven decisions using a scalable and automated Azure data pipeline.

Dashboard

---

Key Features

• Automated Data Pipeline:
  • Fetches live flood data using Azure Data Factory.
  • Stores raw data in Azure Blob Storage.
• Data Transformation:
  • Processes and flattens raw JSON using Azure Databricks.
• Interactive Visualization:
  • Visualizes flood alerts, trends, and regions via Power BI.
• Real-Time Insights:
  • Updates automatically with new data for timely decision-making.

---

Technologies Used

Component	Technology
Data Ingestion	Azure Data Factory
Data Storage	Azure Blob Storage
Data Transformation	Azure Databricks
Data Orchestration	Azure Data Factory
Visualization	Power BI

---

Architecture

High-Level Data Flow

1. Data ingestion from the Flood Monitoring API using Azure Data Factory.
2. Raw JSON data stored in Azure Blob Storage (Raw Container).
3. Data transformed and flattened via Azure Databricks.
4. Processed data stored in Azure Blob Storage (Processed Container).
5. Insights visualized through Power BI Dashboards.

---

Project Components

1. Services Overview

A centralized Azure Resource Group containing:

---

2. Blob Storage and Data Transformation

• Blob Storage: Stores raw and processed data.
• Databricks: Performs data cleansing and flattening.

---

3. Databricks Transformations

• Custom logic for transforming JSON data into a tabular format for analysis.

---

4. Azure Data Factory Pipeline

• Automates the end-to-end data flow:
  • Fetches data from the API.
  • Loads raw data into Blob Storage.
  • Triggers Databricks notebooks for data transformation.
  • Outputs processed data to Blob Storage.

---

5. Power BI Dashboard

Visualizations

• Interactive Maps:

  • Flood Locations: Shows regions affected by flooding.
  • Severity Levels: Highlights severity variations across regions.

• Severity Breakdown:

  • Bar chart illustrating the distribution of severity levels per region.

• Key Metrics:

  • Total flood alerts.
  • Most affected regions.
  • Recent severity changes.