Bright Data ChatGPT Search Scraper (Node.js)

Open in CodeSandbox, sign in with GitHub, then fork the repository to begin making changes.

This project provides a simple Node.js boilerplate for scraping ChatGPT AI search results using the Bright Data Web Scraper API.

---

Table of Contents

• Overview
• Features
• Prerequisites
• Installation
• Usage
• Configuration
• Examples
• Output
• Support
• License

---

Overview

This repository demonstrates how to use the Bright Data Scraper API to trigger and download ChatGPT AI search results. It includes sample prompts and utility functions for batch and custom searches.

---

Features

• Trigger ChatGPT AI searches via Bright Data Scraper /trigger API endpoint
• Monitor progress using /status endpoint
• Download and save results as JSON

---

Prerequisites

• Node.js v16 or higher
• Bright Data account with API KEY

---

Installation

git clone https://github.com/your-org/bright-data-scrape-chatgpt-search-nodejs-project.git
cd bright-data-scrape-chatgpt-search-nodejs-project
npm install

---

Usage

1. Set your Bright Data API token

   Edit index.js and set your API token:

   const API_TOKEN = 'YOUR_API_TOKEN_HERE';

2. Run the scraper

   node index.js

   Results will be saved as a timestamped .json file in the project directory.

---

Configuration

• API Token:
  Get your API token from your Bright Data dashboard under Account Settings.

• Dataset ID:
  The default dataset ID for ChatGPT AI Search is already set in index.js.

---

Examples

Run Sample Searches

By default, the script runs three sample prompts:

const SAMPLE_SEARCHES = [
  { url: "https://chatgpt.com/", prompt: "Top hotels in New York", country: "" },
  { url: "https://chatgpt.com/", prompt: "What are the biggest business trends to watch in the next five years?", country: "" },
  { url: "https://chatgpt.com/", prompt: "Best practices for remote team management", country: "" }
];

Custom Search

Uncomment and edit the following in index.js to run your own prompt:

const customSearch = [createSearch("Best programming languages to learn in 2024")];
const customResults = await searchChatGPT(customSearch);
await saveResults(customResults, 'custom_search.json');

Batch Search

const multipleSearches = [
  createSearch("Climate change solutions"),
  createSearch("AI ethics guidelines"),
  createSearch("Sustainable business practices")
];
const multiResults = await searchChatGPT(multipleSearches);
await saveResults(multiResults, 'multiple_searches.json');

---

Output

• Results are saved as JSON files (e.g., chatgpt_results_YYYY-MM-DDTHH-MM-SS.json).
• Each file contains the raw API response from Bright Data.

---

Support

• Bright Data Help Center
• Contact Support

---

License

This project is licensed under the MIT License. See LICENSE for details.