Skip to content

update the smartcrawler to crawl #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 14, 2025
Merged

update the smartcrawler to crawl #19

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions services/smartcrawler.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ from scrapegraph_py import Client

client = Client(api_key="your-api-key")

response = client.smartcrawler(
response = client.crawl(
Copy link

@cursor cursor bot Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Fix Inconsistent JavaScript Documentation

The Python examples were updated to use client.crawl(), but the JavaScript Quick Start still imports and calls smartCrawler(). This creates an inconsistency where line 127 correctly uses crawl() for JavaScript, but lines 40 and 49 still use the old smartCrawler() function name, making the documentation inconsistent across examples.

Fix in Cursor Fix in Web

url="https://scrapegraphai.com/",
prompt="Extract info about the company",
depth=2,
Expand Down Expand Up @@ -294,7 +294,7 @@ class CompanyData(BaseModel):
description: str = Field(description="Description")
features: list[str] = Field(description="Features")

response = client.smartcrawler(
response = client.crawl(
Copy link

@cursor cursor bot Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Standardize JavaScript Example Crawl Function

The JavaScript schema example still imports and calls smartCrawler() while the corresponding Python example was updated to use client.crawl(). This is inconsistent with the JavaScript markdown mode example at line 127 which correctly uses crawl(), creating confusion about the correct function name to use.

Fix in Cursor Fix in Web

url="https://example.com",
prompt="Extract company info",
schema=CompanyData,
Expand Down Expand Up @@ -330,7 +330,7 @@ from scrapegraph_py import AsyncClient

async def main():
async with AsyncClient(api_key="your-api-key") as client:
task = await client.smartcrawler(
task = await client.crawl(
url="https://scrapegraphai.com/",
prompt="Extract info about the company",
depth=2,
Expand Down