Skip to content

Feature Request: Add Native Support for Wellfound (AngelList) #378

Description

@Vivek02Sharma

@speedyapply

Problem:
JobSpy currently lacks support for Wellfound, a major platform for startup jobs. Historically, scraping Wellfound has been difficult because they use strict DataDome anti-bot protection that blocks standard HTTP requests with 403 Forbidden errors.

Proposed Solution:
I have built a native WellFound module to solve this and will be submitting a PR.

Here is how the implementation works:

  1. Bypassing DataDome: Uses curl_cffi with impersonate="chrome124" to automatically generate authentic TLS signatures and browser headers, successfully passing DataDome checks.
  2. Robust Extraction: Instead of fragile HTML scraping, it extracts the embedded Next.js <script id="__NEXT_DATA__"> payload and parses the Apollo GraphQL cache (JobListingSearchResult) to get rich, structured data (exact compensation, remote policies, company info).
  3. Complex Filtering: Dynamically slugifies locations/roles and includes a custom tokenized search engine that filters jobs if keywords match the title, company, or description.
  4. Architecture Integration: Fully adheres to JobSpy's package structure, maps all data to the JobPost schema, and supports proxy rotation.

Context:
I have successfully tested the module locally, scraping 200+ jobs sequentially. I will open a Pull Request shortly with the full code.

Repo: https://github.com/Vivek02Sharma/JobSpy.git
Branch: wellfound

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions