A blazing fast CLI application that processes sitemaps in golang.

YiraBot: Simplifying Web Scraping for All. A user-friendly tool for developers and enthusiasts, offering command-line ease and Python integration. Ideal for research, SEO, and data collection.

A versatile Python-based web scraper that extracts content from single URLs or entire sitemaps, organizing data into structured text files. Features include sitemap parsing, content grouping by URL structure, and an easy-to-use command-line interface. Ideal for data extraction, content analysis, and web research tasks.

Go library for parsing Sitemaps

Quick lookup for HTTP Status codes of a Sitemap!

A Python script to automate URL update and deletion notifications to Google using the Indexing and Search Console APIs. It fetches URLs from an XML sitemap or processes batch URLs from a JSON file, handling Google API authentication and indexing requests.

Crawl sitemap of a given website and export metadata of its pages recursively into CSV format.

🗺️ Harvest URLs and metadata from website sitemaps efficiently with this fast Python tool. Get organized insights for your digital projects.

This repository provides practical examples of website link scraping using Python and Node.js.

Parse robots.txt and sitemaps using dotnet