Distributed crawler powered by Headless Chrome

Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.

a reliable high-level web crawling & scraping framework for Node.js.

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

Sasori is a dynamic web crawler powered by Puppeteer, designed for lightning-fast endpoint discovery.

⬇️ A simple all-in-one CLI tool to download EVERYTHING from a URL (like youtube-dl/yt-dlp, forum-dl, gallery-dl, simpler ArchiveBox). 🎭 Uses headless Chrome to get HTML, JS, CSS, images/video/audio/subtitles, PDFs, screenshots, article text, git repos, and more...

Web crawling and document processing through a usable interface.

⛏ A versatile Web scraper for Node.js

Tutorial for web scraping / crawling with Node.js.

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

ProxyCrawl Node library for scraping and crawling

Crawling Udemy course info and save into JSON format.

Search for Slack Webhooks token publicly exposed on Github

A web page content extractor

b̶̡̪̬͒l̸̰̗̝̀ỏ̷̡̩g̴͇̑g̶̲̱̽͐i̵̹͗n̶̤̥͂̅̆g̴̮̾̅͜ ̷̧͎͆i̷̛͒͜͠n̸̥̺͒ ̶͚͚͊̿͜t̸̺͙̭̆̊̈́ḧ̶̟́̐e̸̱͔̟̓̓͝ ̶̨͔̾͛̑d̵̥̣̏ȧ̷̼̊r̷̰̝̥̅̌͝k̵̟̥̞̉̍͛

🧩 Proposal to allow user scripts like "expand comments", "hide popups", "fill out this form", etc. to be reusable across pure browser environments, puppeteer, playwright, extensions, AI tools, and many other contexts with minimal adjustment.

A Node.js XML DOM, Parser & Stringifier.

2023.11) velog statistics dashboard fullstack