[Bug]: V2 Pi MCP bridge drops image blocks; fix from #13777 remains unmerged #15869

Before submitting

• I searched existing issues and PRs. The same fix exists in closed, unmerged feat(server): add Pi provider #13777; I found no dedicated open V2 follow-up.
• I included a deterministic source-level reproduction and marked its limits.

Area

apps/server

Summary

The V2 Pi MCP bridge drops image content from tool results. This affects preview_snapshot with includeImage:true: the model receives page metadata but not the screenshot pixels.

This was identified and fixed in #13777, which targeted V1 and was closed without merging after the V2 transition. Its description explicitly lists passing MCP images to Pi as a fix that V2 also needs. This report tracks that remaining gap.

Version or commit

Confirmed in current main at 1e2ecbd9758830669684b494d4398f626b0576e0.
The same code is present in our inspected checkout at ad5178a31aac09c93159ea152daf00720ca56358.

Expected behavior

Preserve valid MCP image blocks as Pi image tool-result content. An image-capable model should receive the screenshot when includeImage:true. Explicit text-only results should remain text-only.

Actual behavior

formatMcpContent extracts text and structured content but does not preserve image blocks.

The registered tool's execute returns only:

content: [{ type: "text", text }]

For a result containing text, structured metadata, and an image, the image bytes disappear. Changing the model inside Pi does not restore them.

Steps to reproduce

This isolates the existing conversion helper without starting a browser, provider, or T3 server. Download the pinned source:

curl -fsSL https://raw.githubusercontent.com/pingdotgg/t3code/1e2ecbd9758830669684b494d4398f626b0576e0/apps/server/src/orchestration-v2/Adapters/piT3McpExtensionSource.ts -o source.ts
bun repro.ts source.ts

Save this as repro.ts:

import assert from "node:assert/strict";
import { readFileSync } from "node:fs";

const source = readFileSync(process.argv[2], "utf8");
const start = source.indexOf("function formatMcpContent(");
const end = source.indexOf("\nfunction isMcpToolError(", start);
assert(start >= 0 && end > start);
const js = new Bun.Transpiler({ loader: "ts" }).transformSync(source.slice(start, end));
const format = new Function(`${js}\nreturn formatMcpContent;`)();
const result = {
  content: [
    { type: "text", text: '{"url":"https://example.com"}' },
    { type: "image", data: "iVBORw0KGgo=", mimeType: "image/png" },
  ],
  structuredContent: { url: "https://example.com" },
};
const returned = { content: [{ type: "text", text: format(result) }] };
assert.equal(returned.content.some((part) => part.type === "image"), false);
assert.equal(JSON.stringify(returned).includes("iVBORw0KGgo="), false);
console.log("Input: text + MCP image + structuredContent");
console.log("Pi tool result: text only; image bytes absent");

The assertions confirm the current defect, not the desired behavior. The image block uses synthetic bytes; this tests conversion, not PNG decoding. The final result construction matches the registered tool's current return shape.

Logs or stack traces

Observed output:

Input: text + MCP image + structuredContent
Pi tool result: text only; image bytes absent

Impact

Pi cannot directly inspect screenshots returned by T3's MCP tools. It can still use text metadata and browser actions. This does not prevent users from viewing saved screenshots or recordings in chat.

Existing fix

#13777 replaces formatMcpContent with mcpToolContent, preserves image blocks, and adds tests:

• sends mirrored structured content once and keeps image blocks
• falls back to structured content when a result has no text

That PR is closed and has no merge commit. The maintainer's closure comment asks for remaining gaps to be submitted against current main. The relevant patch was in apps/server/src/provider/piT3McpExtensionSource.ts; V2 owns it under orchestration-v2/Adapters/.

Environment and verification limits

Our installation uses Debian Linux, T3 0.0.46-nightly.20261004.2657, and Pi 1.0.2. Verification here used the pinned current-main source and Bun, not a live screenshot turn. No end-to-end browser reproduction or native-harness comparison is claimed.

Workaround

Call preview_snapshot with save:true and includeImage:false, then use Pi's image-capable read tool on the returned screenshotPath. This makes the screenshot available to an image-capable model through an extra tool call.