StreetEasy Rental Data Scraper

This project fetches rental data from StreetEasy's GraphQL API, saving both the schema and rental listings to JSON files.

Features

• Fetches GraphQL schema for reference
• Retrieves rental listings with pagination
• Continues fetching until no more data is available
• Saves all data to an organized output directory
• Combines all rental listings into a single file for easy analysis

Setup

1. Make sure you have Node.js installed (v14+ recommended)
2. Install dependencies:

npm install
# or
bun install

Usage

Run the main script to fetch all data:

# Using TypeScript
npx ts-node streeteasy.ts

# Using Bun
bun run streeteasy.ts

Configuration

You can modify the following parameters in the code:

• perPage: Number of listings to fetch per page (default: 100)
• filters: Filter criteria for the search (default: rented listings in area 402)
• sorting: How to sort the results (default: by most recent changes)

Output

All data is saved to the outputs directory:

• schema.json: The GraphQL schema
• rentals_page_X.json: Individual page results
• all_rentals.json: Combined results from all pages

Notes

• The API might have rate limits, so consider adding delays between requests if needed
• The user search token is randomized for each request to avoid tracking