Official SDKs for interacting with the ScrapeGraph AI API - a powerful web scraping and data extraction service.
bash pip install scrapegraph-py
- Web Scraping (basic and structured)
- Credits checking
- Feedback submission
- API status checking
- Local HTML scraping support
- Pydantic schema integration
python from scrapegraph_py import ScrapeGraphClient, smart_scraper from dotenv import load_dotenv import os load_dotenv() api_key = os.getenv("SCRAPEGRAPH_API_KEY") client = ScrapeGraphClient(api_key) url = "https://example.com" prompt = "What does the company do?" result = smart_scraper(client, url, prompt) print(result)
python from pydantic import BaseModel, Field class CompanyInfoSchema(BaseModel): company_name: str = Field(description="The name of the company") description: str = Field(description="A description of the company") main_products: list[str] = Field(description="The main products of the company") result = smart_scraper( client=client, url="https://example.com", prompt="Extract company information", schema=CompanyInfoSchema ) bash npm install scrapegraph-js
- Smart web scraping
- Credits management
- Feedback submission
- Schema-based extraction
- Promise-based API
javascript import { smartScraper, credits, feedback } from 'scrapegraph-js'; const apiKey = process.env.SCRAPEGRAPH_API_KEY; const url = 'https://example.com'; // Basic scraping const result = await smartScraper(apiKey, url, "What does the company do?"); console.log(JSON.parse(result)); // Check credits const creditsInfo = await credits(apiKey); console.log(JSON.parse(creditsInfo));
avascript const schema = { title: "CompanyInfo", properties: { company_name: { type: "string", description: "The name of the company" }, description: { type: "string", description: "A description of the company" }, main_products: { type: "array", items: { type: "string" }, description: "The main products of the company" } }, required: ["company_name", "description"] }; const result = await smartScraper(apiKey, url, "Extract company information", schema);
Both SDKs support authentication via API key. We recommend storing your API key in environment variables:
bash
For Python
export SCRAPEGRAPH_API_KEY="your-api-key-here"
For Node.js
export SCRAPEGRAPH_API_KEY="your-api-key-here"
Or using a .env file:
plaintext
SCRAPEGRAPH_API_KEY="your-api-key-here"
Both SDKs provide consistent error handling json { "error": "HTTP error occurred", "message": "Error details", "status_code": 400 }
- Python 3.9+
- Rye for dependency management (optional)
- Node.js 14+
- npm or yarn
We welcome contributions to both SDKs! Please check our Contributing Guidelines for more information.
Both SDKs are licensed under the MIT License.
For support:
- Visit ScrapeGraph AI Documentation
- Check the examples in the respective SDK's examples directory
- Contact our support team
- Python SDK Documentation
- JavaScript SDK Documentation This README combines information from both SDKs and provides a comprehensive overview of their features and usage. I referenced the following code blocks for accuracy: