Skip to content

The Notion client waits out a rate limit instead of failing the sync - #269

Merged
WaylandYang merged 1 commit into
devfrom
fix/notion-waits-out-the-rate-limit
Sep 3, 2026
Merged

WaylandYang merged 1 commit into
devfrom
fix/notion-waits-out-the-rate-limit

Conversation

@WaylandYang

Copy link
Copy Markdown
Contributor

Part of #215. Thanks to @jeremycapps for running the connector against a real workspace.

Symptom

On a workspace with more than a hundred pages the sync ran for about three minutes and then died with notion search 回了 429 Too Many Requests. The small-workspace run in the same report passed, so the connector itself works; it is the pace that breaks.

Cause

Requests were sequential but not paced. Notion allows roughly three requests per second; a 100 ms response followed immediately by the next request is ten. And a 429 was treated like any other failure: the sync bailed instead of waiting out the Retry-After header Notion sends with it.

Change

A small Paced client wraps every Notion call in notion.rs:

  • at least 350 ms between requests, which puts a 500-page sync at a little over three minutes worst case
  • on 429, wait for Retry-After (seconds, fractional allowed; 1 s if missing or unreadable; capped at 30 s) and retry, up to five times, then surface the error
  • error text is English, since it lands in the source's sync status; the previous message leaked a Chinese word into the reporter's terminal

The extraction path's backoff is not reused on purpose: it is tuned for model providers and their error taxonomy. This one is local to the connector and about twenty lines.

Verification

  • cargo fmt, cargo clippy --all-targets -D warnings, cargo test -p utopia-server notion: clean, 7 passed (a unit test pins the Retry-After parsing: header value, fractional, missing, garbage, negative, capped)
  • The live test needs a real token and is skipped here. @jeremycapps, if you can run UTOPIA_NOTION_TEST_TOKEN=... cargo test -p utopia-server notion once more on the large workspace after this lands, that would confirm both the pacing and pagination past 100.

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
@WaylandYang
WaylandYang merged commit f366693 into dev Sep 3, 2026
@WaylandYang
WaylandYang deleted the fix/notion-waits-out-the-rate-limit branch September 3, 2026 17:28
@jeremycapps

Copy link
Copy Markdown
Contributor
CleanShot 2026-09-05 at 19 30 48@2x

just reran it on the latest dev branch and all tests passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants