Scrapious

A multi-purpose web scraper that crawls websites and extracts all discovered links with configurable depth control.

Features

• Crawls a starting URL and discovers all links on the page
• Configurable crawl depth (follow links within the same domain)
• Filters external vs internal links
• Outputs results to a text file
• Simple CLI interface

Installation

git clone https://github.com/ShantanuRauthan/Scrapious.git
cd Scrapious
pip install -r requirements.txt

Usage

python3 main.py <url>

The scraper will crawl the provided URL, extract all links, and save them to a text file in the same directory.