🛠️ Python Project for Data Engineering

Overview

This repository contains my completed project for the Python Project for Data Engineering course offered by Coursera. The course focuses on building foundational ETL (Extract, Transform, Load) pipelines using Python, and this project demonstrates the practical application of those skills.

---

🚀 Project Goals

• ✅ Extract structured data from a public web source
• ✅ Transform raw data into clean, analyzable formats
• ✅ Load the final dataset into both a CSV file and a SQLite database
• ✅ Run SQL queries to validate the pipeline
• ✅ Log each step of the ETL process for traceability

---

📂 Repository Structure

├── etl_project_gdp.py          # Main ETL script
├── Countries_by_GDP.csv        # Output CSV file
├── World_Economies.db          # SQLite database
├── Countries_by_GDP.json       # Output JSON file
├── etl_project_log.txt         # Execution log
├── README.md                   # Project documentation

---

🧠 Skills Demonstrated

• Web scraping with BeautifulSoup
• Data cleaning and transformation with Pandas
• Exception handling and logging
• SQL integration using SQLite
• Modular Python scripting for ETL workflows

---

📊 Output Example

The final dataset includes:

Country	GDP_USD_billion
United States	26854.60
China	17700.00
Japan	4937.00

Only countries with GDP > 100 billion USD are included in the final query output.

---

🧪 How to Run

1. Clone the repository
2. Install dependencies:

   pip install requests beautifulsoup4 pandas numpy

3. Run the ETL script:

   python etl_project_gdp.py

---

📚 Course Link

This project was developed as part of the Python Project for Data Engineering specialization by IBM on Coursera.