Skip to content

Add Wellfound job board scraper support and update documentation - #379

Open
Vivek02Sharma wants to merge 2 commits into
speedyapply:mainfrom
Vivek02Sharma:wellfound
Open

Add Wellfound job board scraper support and update documentation#379
Vivek02Sharma wants to merge 2 commits into
speedyapply:mainfrom
Vivek02Sharma:wellfound

Conversation

@Vivek02Sharma

Copy link
Copy Markdown

Description

This PR adds native support for scraping startup job postings from Wellfound (formerly AngelList Talent), expanding JobSpy's supported platforms.

Closes #378

Core Changes:

  • Wellfound Scraper Integration (jobspy/wellfound/):
    • Added WellfoundScraper that bypasses DataDome anti-bot protection using curl_cffi for Chrome TLS impersonation.
    • Parses highly structured data (compensation, remote policy) directly from Wellfound's Apollo GraphQL cache instead of fragile HTML parsing.
    • Implements round-robin proxy rotation and a tokenized keyword search engine.
  • Library Updates (jobspy/):
    • Registered Site.WELLFOUND in the main scraper mapping and enum.
    • Added WellfoundException for precise error handling.
  • Documentation (README.md):
    • Added Wellfound to the list of supported boards.
    • Added FAQ guidance on bypassing 403 DataDome errors via proxies.

Testing:

  • Passed pre-commit and black formatting checks.
  • Locally verified successful extraction of 200+ jobs sequentially into the standard JobPost schema.

Copilot AI lite review requested due to automatic review settings August 6, 2026 06:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Add Native Support for Wellfound (AngelList)

2 participants