Repository navigation
feat: agents can show HTML pages inline in threads - #278
Conversation
Port of T3 Code's inline HTML renders (pingdotgg/t3code#15968, with the #16196 MCP Apps bridge and #16283 height fix). Agents get html_preview and html_render MCP tools; published pages render in a sandboxed frame above the agent's reply on web, desktop, and iOS. Pages are for the user, so subagents and orchestrator workers do not get the tools. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d7768104a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| payload: object, | ||
| invocation: McpInvocationContext.McpInvocationScope, | ||
| runtimeContext: Context.Context<never>, | ||
| signal?: AbortSignal, |
There was a problem hiding this comment.
For HTML calls, cancellation notifications, request disconnects, and handler shutdown only abort the signal passed here, but Effect.runPromiseWith(runtimeContext) is invoked without { signal }. The tool effect therefore keeps running: html_preview retains its browser and proxy until its own deadline, the newly added Effect.never cancellation tests hang, and html_render can continue publishing after its MCP request was cancelled. Pass the signal to the Effect runner so interruption reaches the tool and its finalizers.
Useful? React with 👍 / 👎.
| id: `effect:${command.commandId}:attachment.cleanup`, | ||
| commandId: command.commandId, | ||
| threadId: command.threadId, | ||
| request: { type: "attachment.cleanup", attachmentIds, htmlRenderThreadId: thread.id }, |
There was a problem hiding this comment.
When a source conversation has a surviving fork, the fork's inherited visible items still contain the source's HTML attachment IDs, but this deletion effect sweeps every HTML file minted by the source thread. Deleting the original conversation therefore removes files that the fork still displays, turning all inherited visual replies into broken asset URLs. Cleanup needs to retain or copy pages referenced by live forks, or remove them only after the last referencing thread is deleted.
Useful? React with 👍 / 👎.
| const files = yield* Effect.forEach( | ||
| [...new Set(references.map((reference) => reference.path))], | ||
| (path) => | ||
| fileSystem.stat(filePathFor(path)).pipe( |
There was a problem hiding this comment.
When a provider is running with read-only or workspace-scoped filesystem access, it can still pass any absolute image path to html_preview or html_render, and this code has the unrestricted server process stat and read it without checking the thread workspace or runtime permissions. Preview returns the rendered pixels immediately, while publish embeds the bytes into a script-capable page, so a known outside-workspace PNG/SVG—or a symlink to one from a writable workspace—bypasses the provider's filesystem isolation and can disclose the file. Resolve images through an authorized workspace root or reject paths outside the caller's permitted scope.
Useful? React with 👍 / 👎.
| const LOCAL_IMAGE_PATTERN = new RegExp( | ||
| String.raw`(["'\x60])(${ABSOLUTE_PATH}(?:(?!\1)[^\r\n]){0,2048}?\.(?:${IMAGE_EXTENSIONS}))\1` + | ||
| String.raw`|url\(\s*(${ABSOLUTE_PATH}[^\s"'\x60()]{0,2048}?\.(?:${IMAGE_EXTENSIONS}))\s*\)`, | ||
| "gid", |
There was a problem hiding this comment.
On Windows, the usual absolute path such as C:\Users\me\chart.PNG does not match this case-sensitive pattern: the drive expression only accepts [a-z], and the generated extension alternatives are lowercase. The reference is consequently omitted from both inlining and missingImages, so preview and publish report success while the client receives a broken image URL. Make this pattern case-insensitive, which also handles valid uppercase CSS URL(...) spellings.
Useful? React with 👍 / 👎.
1 participant
Agents can only answer in Markdown, so charts, tables, diagrams, image collages, and mockups are out of reach. T3 Code just shipped inline HTML renders (pingdotgg/t3code#15968). This ports it to Pathway, including the follow-ups for the MCP Apps bridge (#16196) and the height and scroll fix (#16283).
What changes for users
An agent builds one self-contained HTML page, checks it with
html_preview(a screenshot), then publishes it withhtml_render. The page appears inline above the agent's reply on web, desktop, and iOS, and stays visible when the turn folds into Worked for…. Each page:Pages are for the user, so subagents and orchestrator workers never get these tools. They aren't listed for those threads, and calls are refused with a message telling the agent to report back to its parent. Forks keep the tools.
The existing
visualize{}cards are unchanged.How it works
Server. Two new MCP tools.
file://and other local files are refused.disposition: "inline". Inline.htmlresponses carryContent-Security-Policy: sandbox allow-scripts allow-forms allow-popups, plusnosniffandno-referrer. URLs resolve against the owning environment, so remote and Pathway Connect work the same way.Web and desktop.
HtmlRenderFrameandHtmlRenderDocument. The iframe sandbox isallow-scripts allow-forms.frame-src http: https:.@spiritdevs/shared/themePalettes.iOS (native Swift).
WKWebViewwith a non-persistent store and no file access. The page must be served as HTML with a sandbox CSP.Docs.
docs/user/html-renders.mddocs/internals/html-renders.mdVerification
build-for-testingpasses, and 28 focused tests pass, including WebKit tests that load real sandboxed pages.OrchestratorMcpService.ts:810diagnostic, which this PR doesn't touch.Each package was reviewed by the other model, and the confirmed findings were fixed. The notable ones:
/\host\shareimage path could make the server open an SMB connection.Not yet verified:
HtmlPreviewBrowser.integration.test.tsis written but gated behindPATHWAY_HTML_RENDER_BROWSER_TESTS=1and hasn't been run. It covers local-file leaks, private-network access, popups, and WebRTC.Known limits / follow-ups
Implemented and reviewed by Claude Opus 5.5 and GPT-6.1 Sol subagents, orchestrated by Claude Opus 5.5 in Claude Code via Pathway.
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.