Skip to content

refactor(web-fetch): render through the canonical browser interface (#13236) - #13309

Merged
mrveiss merged 2 commits into
Dev_new_guifrom
issue-13236-webfetch
Aug 2, 2026
Merged

mrveiss merged 2 commits into
Dev_new_guifrom
issue-13236-webfetch

Conversation

@mrveiss

@mrveiss mrveiss commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Closes #13236

Partial — #13236 must stay OPEN. The close keyword is required by the
Validate PR issue link gate; merging to Dev_new_gui does not auto-close.
This is step 2 of 5. Steps 3–5 (content_reach, tool_handler,
api/playwright) remain, and #13204 closes only with step 4.

Thinking Path

The first caller migrated off a directly-named stack, now that the five
contract gaps the audit exposed are closed.

_fetch_playwright imported services.playwright_service directly. It now
declares what it actually needs and lets dispatch resolve it:

requires={Capability.EXTRACT_HTML, Capability.OUT_OF_PROCESS}

Both requirements are load-bearing, not decoration. The worker and the
container both satisfy out-of-process extraction, and register_all
registers the worker first. Under the pre-#13306 single EXTRACT
capability this caller would have resolved to the worker and received
get_text output — plain text where a parser expects HTML, and nothing
would have raised
, because text is a valid content value.

That is why this migration went second rather than first, and why the format
split had to land before it.

What Changed

autobot-backend/web_fetch/fetcher.py — _fetch_playwright resolves a
browser by capability instead of importing a stack, and passes its timeout
through ExtractRequest.timeout_seconds.

The inline SSRF check is kept, not deferred to the registry's guard. The
container has network reach to private IPs, so one extra DNS resolution is
cheaper than depending on a single chokepoint — and the docstring now says so,
to stop a later reader deleting it as redundant.

fetch_playwright_interface_test.py — 6 tests.

Verification

autobot-backend/tests/unit/web_fetch
  + tests/test_untrusted_web_content_boundary.py
  + autobot-backend/web_fetch                       186 passed
autobot_shared/browser + browser_backends            46 passed
bandit                                                0 issues
isort · black --line-length=120 · flake8              clean

The tests pin behaviour-preservation rather than just "it runs":

  • it lands on the container, the same stack it called before;
  • the render payload is unchanged — same endpoint, same wait: networkidle;
  • the caller's timeout still reaches the endpoint, in milliseconds;
  • the worker is never selected, asserted end-to-end from this caller
    despite it registering first — the exact silent swap this prevents;
  • a non-public URL never reaches a backend;
  • failure still returns None, so the render cascade falls through to the
    next step rather than propagating.

What this does not do

Model Used

Claude Opus 5 (1M context)

…13236)

ADR-009 step 2 -- the first caller migrated off a directly-named stack, now
that the five contract gaps the audit found are closed.

_fetch_playwright imported services.playwright_service directly. It now
declares what it actually needs and lets dispatch resolve it:

  requires={Capability.EXTRACT_HTML, Capability.OUT_OF_PROCESS}

EXTRACT_HTML because the result feeds a parser, so text will not do.
OUT_OF_PROCESS because the entire reason this fallback exists is that
Playwright must not run inside the backend's own process.

Behaviour-preserving, and that is not incidental. Both requirements are
load-bearing: the worker and the container BOTH satisfy out-of-process
extraction, and register_all registers the worker first, so under the
pre-#13306 single EXTRACT capability this caller would have resolved to the
worker and received get_text output -- plain text where a parser expects
HTML, with nothing raising, because text is a valid result shape.

The inline SSRF check is KEPT rather than deferred to the registry's guard.
It is deliberate defence in depth: the container has network reach to
private IPs, and one extra DNS resolution is cheaper than depending on a
single chokepoint. The docstring says so, so a later reader does not remove
it as redundant.

Six tests pin that this lands on the same stack it called before, sends the
same render payload with the same wait, forwards the caller's timeout, keeps
returning None on failure so the render cascade can fall through, and that
the worker is never selected despite registering first.

186 tests pass across autobot-backend/tests/unit/web_fetch,
test_untrusted_web_content_boundary.py and autobot-backend/web_fetch.
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!

@mrveiss
mrveiss merged commit 9ddc811 into Dev_new_gui Aug 2, 2026
47 of 49 checks passed
@mrveiss
mrveiss deleted the issue-13236-webfetch branch August 2, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant