Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
The unix-way web crawler
🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
Golang Crawling and scraping framework
Interactive CLI Web Crawler
GoScrapy: Harnessing Go's power for blazingly fast web scraping, inspired by Python's Scrapy framework.
Fast, highly configurable, cloud native dark web crawler.
A tool to find redirection chains in multiple URLs
Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.
Spider - web crawler and local wordlist processor to generate frequency sorted wordlist / ngrams
Via Text Density Simple Web Crawler With Go
A simple proxy server to integrate crawl4ai with OpenWebUI
豆备 (Doubak) - 豆瓣 (Douban.com) 命令行版备份工具，备份完整页面镜像，Golang实现。
A web crawler built in the Go programming language
Scrape emails from a website using recursive crawling, the best anti-obfuscation techniques, and validate all addresses before saving to a file.
Distributed DNS + HTTP crawler with queueing, deduplication, tracing, and ingest batching.
Add a description, image, and links to the web-crawler topic page so that developers can more easily learn about it.
To associate your repository with the web-crawler topic, visit your repo's landing page and select "manage topics."