Repository navigation
refactor(web-fetch): render through the canonical browser interface (#13236) - #13309
Merged
Merged
Conversation
…13236) ADR-009 step 2 -- the first caller migrated off a directly-named stack, now that the five contract gaps the audit found are closed. _fetch_playwright imported services.playwright_service directly. It now declares what it actually needs and lets dispatch resolve it: requires={Capability.EXTRACT_HTML, Capability.OUT_OF_PROCESS} EXTRACT_HTML because the result feeds a parser, so text will not do. OUT_OF_PROCESS because the entire reason this fallback exists is that Playwright must not run inside the backend's own process. Behaviour-preserving, and that is not incidental. Both requirements are load-bearing: the worker and the container BOTH satisfy out-of-process extraction, and register_all registers the worker first, so under the pre-#13306 single EXTRACT capability this caller would have resolved to the worker and received get_text output -- plain text where a parser expects HTML, with nothing raising, because text is a valid result shape. The inline SSRF check is KEPT rather than deferred to the registry's guard. It is deliberate defence in depth: the container has network reach to private IPs, and one extra DNS resolution is cheaper than depending on a single chokepoint. The docstring says so, so a later reader does not remove it as redundant. Six tests pin that this lands on the same stack it called before, sends the same render payload with the same wait, forwards the caller's timeout, keeps returning None on failure so the render cascade can fall through, and that the worker is never selected despite registering first. 186 tests pass across autobot-backend/tests/unit/web_fetch, test_untrusted_web_content_boundary.py and autobot-backend/web_fetch.
Contributor
✅ SSOT Configuration Compliance: Passing🎉 No hardcoded values detected that have SSOT config equivalents! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #13236
Thinking Path
The first caller migrated off a directly-named stack, now that the five
contract gaps the audit exposed are closed.
_fetch_playwrightimportedservices.playwright_servicedirectly. It nowdeclares what it actually needs and lets dispatch resolve it:
Both requirements are load-bearing, not decoration. The worker and the
container both satisfy out-of-process extraction, and
register_allregisters the worker first. Under the pre-#13306 single
EXTRACTcapability this caller would have resolved to the worker and received
get_textoutput — plain text where a parser expects HTML, and nothingwould have raised, because text is a valid
contentvalue.That is why this migration went second rather than first, and why the format
split had to land before it.
What Changed
autobot-backend/web_fetch/fetcher.py—_fetch_playwrightresolves abrowser by capability instead of importing a stack, and passes its timeout
through
ExtractRequest.timeout_seconds.The inline SSRF check is kept, not deferred to the registry's guard. The
container has network reach to private IPs, so one extra DNS resolution is
cheaper than depending on a single chokepoint — and the docstring now says so,
to stop a later reader deleting it as redundant.
fetch_playwright_interface_test.py— 6 tests.Verification
The tests pin behaviour-preservation rather than just "it runs":
renderpayload is unchanged — same endpoint, samewait: networkidle;despite it registering first — the exact silent swap this prevents;
None, so the render cascade falls through to thenext step rather than propagating.
What this does not do
tool_handler), whichis the production behaviour change: the guard starts rejecting internal URLs
that
send_to_browser_vmaccepts today. That step needs the existingis_url_allowedallowlist call sites enumerated first, and your sign-off.content_reach(step 3) is next and now unblocked —BrowserResult.structuredexists for it.
Model Used
Claude Opus 5 (1M context)