Skip to content

Latest commit

 

History

History
 
 

README.md

HeadlessX Logo

The World's Most Advanced Anti-Detection Scraping Full Stack Web App


Version Release Detection Status


Node.js Next.js Camoufox TypeScript Playwright


License PRs Welcome Maintenance Made with Love


🎯 0% Detection Rate on all major anti-bot systems

Powered by Camoufox — Firefox with C++ level fingerprint spoofing

📖 Documentation🚀 Quick Start🐛 Report Bug✨ Request Feature


🆕 What's New in V2.0

🦊 Camoufox Engine 🛡️ 0% Detection 3x Faster
Firefox with binary-level stealth patches Passes CreepJS, Sannysoft, & all major tests Optimized context pooling

V2.0 Changelog

+ 🦊 NEW: Camoufox browser engine (replaces Chromium + stealth plugins)
+ 🛡️ NEW: 0% headless detection rate (was 67%+)
+ 🔒 NEW: Built-in WebRTC leak protection
+ 🎭 NEW: C++ level Canvas/WebGL/AudioContext fingerprint spoofing
+ ⚡ IMPROVED: Faster browser launches with persistent contexts
- 🧹 REMOVED: playwright-extra (no longer needed)
- 🧹 REMOVED: puppeteer-extra-plugin-stealth (no longer needed)

📊 Detection Benchmark Comparison

Tool Headless Detection Stealth Score Engine
🦊 HeadlessX V2.0 (Camoufox) 0% 0% Firefox (C++ patched)
HeadlessX V1 (Playwright Extra) ❌ 67% ⚠️ 50% Chromium (JS patches)
Puppeteer Stealth ❌ 33% ❌ 80% Chromium (JS patches)
Vanilla Playwright ❌ 100% ❌ 100% Chromium
Vanilla Selenium ❌ 100% ❌ 100% Chrome

Lower scores = better (0% = undetectable)


🤖 Scrapers

Google SERP 🌐 Website Scraping
Extract search results with zero detection Scrape any website with full JS rendering
Live Live

Coming Soon

Amazon LinkedIn Twitter Instagram
Product & Reviews Job Listings & Profiles Tweets & Trends Posts & Stories

🖥️ UI Screenshots

Google SERP Scraper

Google SERP UI

Website Scraper

Website Scraper UI


📸 Proof of Undetectability

BrowserScan (100% Passed)

BrowserScan

Pixelscan (Human Detection) Profile Tested with Proxy
Pixelscan Proxy Test

📋 Table of Contents


⚡ Quick Start

Prerequisites:

  • Node.js 18+ (v22+ recommended)
  • pnpm 9+ (npm install -g pnpm)
  • PostgreSQL Database — either:
    • Supabase (recommended, free tier available)
    • Self-hosted PostgreSQL 14+

1️⃣ Clone & Configure Environment

git clone https://github.com/your-repo/HeadlessX.git
cd HeadlessX

# Copy environment template
cp .env.example backend/.env

Edit backend/.env with your database connection:

# REQUIRED: Your PostgreSQL connection string
DATABASE_URL="postgresql://user:password@host:5432/database"

# Optional: Server configuration
PORT=3001
NODE_ENV=development

💡 Supabase Users: Find your connection string at:
Dashboard → Settings → Database → Connection String (URI)

2️⃣ Install Dependencies

# Install all packages (workspace: root, backend, frontend)
pnpm install

⚠️ If you see a Prisma warning during install, that's normal!
It means you need to configure DATABASE_URL first, then run pnpm db:push.

3️⃣ Install AI Models (CAPTCHA Solving)

Windows Linux / macOS
install.bat
chmod +x install.sh
./install.sh

4️⃣ Setup Database

# Push Prisma schema to your database
pnpm db:push

# Or use migrations for production
pnpm db:migrate

5️⃣ Start Development Server

# Start both frontend and backend concurrently
pnpm dev

Or start individually:

Windows Linux / macOS
scripts\start.bat
./scripts/start.sh

6️⃣ Access the Application

Service URL Notes
🖥️ Dashboard http://localhost:3000 Frontend UI
🔗 Backend API http://localhost:3001 REST API

Custom Ports

You can customize ports via environment variables:

# Backend (from root)
cross-env PORT=4001 pnpm dev:backend

# Frontend (Linux/macOS)
PORT=4000 pnpm --filter headlessx-frontend dev

🔥 Key Features

🦊 Camoufox Stealth Engine (V2.0)

Feature Description
🔒 Binary-Level Patches Firefox modified at C++ level
🎭 Fingerprint Spoofing Canvas, WebGL, AudioContext
🌐 WebRTC Protection No IP leaks in headless mode
🖱️ Human Behavior Natural mouse movements & scrolling
📱 Device Emulation Realistic viewport & user agents
🚫 Zero Detection Passes all anti-bot tests

🖥️ Modern Dashboard

Feature Description
⚛️ Next.js 16 React 19 with Turbopack
🎨 HeroUI Beautiful dark/light mode UI
⚙️ Live Config Change settings without restart
📊 Request Logs Full history with screenshots
🧪 Playground Test scraping in real-time
🔑 API Keys Secure key management

🌐 API Endpoints

Website Scraping APIs

Endpoint Method Description
/api/website/html POST Get raw HTML (fast)
/api/website/html-js POST Get HTML with JS rendering
/api/website/content POST Get Markdown content
/api/website/screenshot POST Capture full-page Screenshot
/api/website/stream POST Real-time SSE Stream

Google SERP APIs

Endpoint Method Description
/api/google-serp/search POST Extract Google search results
/api/google-serp/stream GET Real-time SSE Stream

Example Request

curl -X POST http://localhost:3001/api/website/html \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-api-key" \
  -d '{"url": "https://example.com", "stealth": true}'

Example Response

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "title": "Example Domain",
    "html": "<!DOCTYPE html>...",
    "statusCode": 200
  }
}

Configuration

Environment Variables

⚠️ Note: Most configuration (Browsers, Stealth, Proxy, Timeouts) is managed directly via the Dashboard Settings.

Only the following core variables are required in .env:

Variable Default Description
PORT 3001 Backend API port
DATABASE_URL - PostgreSQL connection (Supabase or self-hosted)
NEXT_PUBLIC_API_URL http://localhost:3001 Frontend API URL

Dashboard Settings

Configure these live at /settings:

General Configuration

  • Headless Mode: Run browser without UI (faster execution)
  • Browser Timeout: Max execution time per job (default: 60000ms)
  • Max Concurrent Jobs: Concurrent scraping jobs (recommended: 3-8)

Browser Engine (Camoufox)

  • Block WebRTC: Prevent IP leaks via WebRTC
  • Camoufox GeoIP: Spoof location based on IP
  • Enable Cache: Cache resources for speed

Proxies

  • Manage proxy configurations and rotation settings

🛠️ Manual Setup

Backend

cd backend

# 1. Configure environment
cp ../.env.example .env
# Edit .env with your DATABASE_URL

# 2. Install dependencies
pnpm install

# 3. Download Camoufox browser
npx camoufox-js fetch

# 4. Setup database
npx prisma db push

# 5. Start dev server
pnpm dev

Frontend

cd frontend
pnpm install
pnpm dev

🤝 Contributing

Contributions are welcome! Please read our Contributing Guide for details.

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.


🌟 Star History

If you find HeadlessX useful, please consider giving it a star! ⭐


Built with ❤️ using cutting-edge technologies


Node.js Next.js Firefox Playwright TypeScript React TailwindCSS Prisma


🦊 HeadlessX V2.0 — Undetectable by Design

© 2026 HeadlessX. All rights reserved.