fix(cli): execute network-capable adapters in Node to bypass CORS#45
fix(cli): execute network-capable adapters in Node to bypass CORS#45osen77 wants to merge 1 commit intoepiral:mainfrom
Conversation
Site adapters with `capabilities: ["network"]` make cross-origin API calls (e.g. hackernews/top calls Firebase API from a ycombinator page). These calls fail with "TypeError: Failed to fetch" when executed via CDP Runtime.evaluate in the page context, because the page's CORS/CSP policy blocks cross-origin requests. Fix: detect `capabilities: ["network"]` in adapter metadata and execute the adapter script directly in the Node process using `new Function()`. Node's fetch() has no CORS restrictions, so cross-origin API calls succeed without needing browser context. Adapters without the "network" capability continue to execute via CDP in page context, preserving cookie-based authentication for sites that require login (e.g. zhihu, weibo). Fixes epiral#41
|
Thanks for the contribution! The idea of running pure-API adapters in Node to bypass CORS/CSP is interesting, but there are some fundamental issues with the current approach that need to be addressed. 1.
|
…ble-web-security Site adapters that make cross-origin API calls (e.g. hackernews/top fetching from firebaseio.com, bbc/news fetching from feeds.bbci.co.uk) fail with "TypeError: Failed to fetch" because Runtime.evaluate runs in page context where browser CORS restrictions apply. Two complementary fixes: 1. Add --disable-web-security to managed browser launch args. This flag only affects the dedicated bb-browser Chrome instance (separate user-data-dir), not the user's daily browser. On about:blank tabs (null origin), this completely eliminates CORS enforcement. 2. Route adapters with capabilities: ["network"] to an about:blank tab instead of the target domain tab. Combined with (1), cross-origin fetch() calls succeed without CORS restrictions. Adapters requiring cookies (same-origin) continue to run on the domain tab as before. 3. Auto-retry fallback: if any adapter fails with "Failed to fetch" on a domain tab, automatically retry on an about:blank tab. This catches adapters that make cross-origin calls but don't declare capabilities: ["network"]. Tested: hackernews/top, bbc/news, arxiv/search, stackoverflow/search, wikipedia/summary — all return valid JSON after this fix. Fixes epiral#41 Closes epiral#110 (跨域 fetch 部分) Related: epiral#104, epiral#45 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
capabilities: ["network"](e.g.hackernews/top,v2ex/hot) make cross-origin API calls that fail withTypeError: Failed to fetchwhen executed via CDPRuntime.evaluatein page context, due to CORS/CSP restrictionscapabilities: ["network"]in adapter metadata and execute the script directly in the Node process vianew Function(), wherefetch()has no CORS restrictionsTest plan
bb-browser site hackernews/top 3— returns JSON (was failing with CORS error)bb-browser site v2ex/hot— returns JSON (cross-origin API)bb-browser eval "document.title"— still works via CDP (regression check)bb-browser site zhihu/hot— runs via CDP with login cookies (nonetworkcapability)Fixes #41
🤖 Generated with Claude Code