Data Broker Removal

Automated opt-out tool for removing personal information from data brokers. An Incogni-like service scaffold with automated Playwright workers, a user dashboard, and a REST API.

Data Broker Removal is a monorepo starter for building a full-stack data broker opt-out service. It scaffolds a complete system with automated browser workers, a Fastify REST API, a Next.js user dashboard, and a shared broker catalog — all orchestrated with Docker Compose.

Architecture

The project is structured as a pnpm monorepo with four packages:

Package	Description	Tech
packages/api	Fastify REST API	Node.js, PostgreSQL, BullMQ
packages/worker	Playwright automation workers	Playwright, BullMQ, Redis
packages/dashboard	Next.js user dashboard	Next.js, React, Tailwind
packages/shared	Shared types and schema	TypeScript, PostgreSQL DDL

Infrastructure: PostgreSQL 16 + Redis 7 (via Docker Compose)

Quick Start

# Initialize a new project
bash scripts/init.sh my-broker-removal
cd my-broker-removal

# Start infrastructure
pnpm infra:up

# Run migrations
pnpm db:migrate

# Start API
cd packages/api && pnpm dev

# Start worker (in another terminal)
cd packages/worker && pnpm dev

# Start dashboard (in another terminal)
cd packages/dashboard && pnpm dev

Scripts

init.sh — Scaffold a new project

bash scripts/init.sh <project-name>

Sets up the full monorepo with pnpm workspaces, Turborepo, Docker Compose (PostgreSQL + Redis), database migrations, and a sample broker catalog with 5 entries.

add-broker.sh — Add a broker to the catalog

# Interactive prompt
bash scripts/add-broker.sh

# Or pass JSON directly
bash scripts/add-broker.sh '{"name":"Spokeo","domain":"spokeo.com","method":"form","optOutUrl":"https://www.spokeo.com/opt_out","requiredFields":["fullName","email","address"],"selectors":{"name":"[data-testid=optout-name]","email":"[data-testid=optout-email]","submit":"[data-testid=optout-submit]"},"verificationType":"email_link","region":["US"],"category":"people-search"}'

generate-worker.sh — Generate a Playwright worker

bash scripts/generate-worker.sh <broker-id>

Outputs a Playwright automation script at packages/worker/src/brokers/<broker-id>.ts using the broker's catalog entry.

Project Structure (After Init)

<project-name>/
├── docker-compose.yml          # PostgreSQL 16 + Redis 7
├── turbo.json                  # Turborepo config
├── package.json                # pnpm workspaces
├── packages/
│   ├── shared/                 # TypeScript types, schema, broker catalog
│   │   └── src/
│   │       ├── schema.sql
│   │       ├── types.ts
│   │       └── broker-catalog.json
│   ├── api/                    # Fastify REST API
│   │   └── src/
│   │       ├── server.ts
│   │       ├── db.ts
│   │       ├── routes/         # users, brokers, requests
│   │       └── jobs/           # scheduler
│   ├── worker/                 # Playwright automation
│   │   └── src/
│   │       ├── worker.ts
│   │       ├── scheduler.ts
│   │       ├── browser-pool.ts
│   │       └── brokers/        # Generated per-broker scripts
│   └── dashboard/              # Next.js user dashboard
│       └── src/
│           ├── app/            # App Router pages
│           └── lib/            # API client

Design Decisions

• Monorepo — Separate concerns; workers crash independently from API
• BullMQ over Redis — Retry with backoff, concurrency controls, dead-letter queues
• Playwright in Docker — One container per job to isolate sessions, cookies, and fingerprints
• PostgreSQL — ACID for request state; JSONB for broker configs
• Next.js — Familiar, modern dashboard framework

Requirements

• Node.js 20+
• Docker & Docker Compose
• Playwright (installed automatically)
• pnpm

Environment

Create a .env file with:

DATABASE_URL=postgresql://user:password@localhost:5432/data_broker_removal
REDIS_URL=redis://localhost:6379

License

Private — This project is proprietary and confidential.

Author

Developed by Cameron Ashley / Ashbi Design.