Skip to content

About

A collection of my data engineering projects spanning web scraping, pipeline orchestration, and personal experiments across the data stack.

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

477 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Scraping Projects

A collection of web scraping and data extraction projects built with Python and various scraping libraries.


Technologies Used

Scraping & Automation

  • Selenium — Browser automation and dynamic content scraping
  • SeleniumBase — Enhanced Selenium with anti-bot bypass
  • Undetected ChromeDriver — Stealth browser automation
  • Selenium Driverless — CDP-based browser control without chromedriver
  • Playwright — Modern browser automation
  • Scrapy — Large-scale web crawling and scraping framework
  • Crawlee — Web scraping and crawling library
  • BeautifulSoup — HTML and XML parsing
  • Requests — HTTP requests and REST API interactions
  • curl_cffi — HTTP client with browser fingerprint impersonation

Data & Visualization

  • Pandas — Data manipulation and analysis
  • Streamlit — Interactive data dashboards and visualization

Structure

scraping-projects/
  ├── milestone-projects/
  ├── other-projects/
  ├──.gitignore
  ├──README.md


Note

Some projects in this repository were built for learning and experimentation purposes. All scraping was done responsibly and in accordance with the respective website's terms of service.

About

A collection of my data engineering projects spanning web scraping, pipeline orchestration, and personal experiments across the data stack.

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages