Summary
Zag can already decode PNGs (src/png_decode.zig, src/halfblock.zig, src/buffers/image.zig) and render image-backed tool_result nodes in the conversation tree, but users cannot attach images to their own messages. There is no .image variant in ContentBlock, submitInput is text-only, and onUserInputSubmitted has no image path.
Motivation
Users expect to paste screenshots or diagrams into the conversation and have the model analyze them inline. Today the only workaround is describing the image in text or asking the agent to read a path manually.
Prior art
pi (earendil-works/pi): on clipboard paste, writes the image bytes to a temp file (/tmp/pi-clipboard-<uuid>.png) and inserts the file path directly into the editor text. On submission, image paths are detected by MIME sniffing, optionally resized, converted to base64, and sent as ImageContent blocks alongside the text.
opencode (anomalyco/opencode): in their web client, pasted file attachments are collected separately; in their GitHub agent, image URLs are downloaded, replaced with @filename references in the prompt, and shipped as base64 attachments.
Both converge on the same under-the-hood pattern: save to a temp path, insert the path into the prompt text, then detect image paths at send-time and convert them to base64 content blocks for the LLM.
Proposed behavior
- Paste handling: when a terminal paste event contains image bytes (or a future drag-and-drop flow lands), write the bytes to a temp file and insert the absolute path into the draft at the cursor.
- Send-time detection: before a user message is submitted to the agent, scan the text for image file paths (by extension or MIME sniff), read/convert them to base64, and strip or keep the path reference in the text depending on what works best for the model.
- Wire format: add an
.image variant to ContentBlock (or reuse the existing ImageBuffer / BufferRegistry flow) so Anthropic and OpenAI providers can serialize it correctly.
- Persistence:
Session.zig JSONL should store image references (paths or base64) so replay and resume work.
Scope
Out of scope
- Native terminal image protocols (iTerm2, kitty, sixel)
- Image generation by the model
- Automatic image resizing / compression (can be fast-follow)
Relevant files
src/types.zig — ContentBlock union
src/AgentRunner.zig — submitInput
src/EventOrchestrator.zig — onUserInputSubmitted, paste event handling
src/Conversation.zig — persistUserMessage, node tree
src/providers/anthropic.zig, src/providers/openai.zig — wire serialization
src/buffers/image.zig, src/BufferRegistry.zig — existing image buffer infra
src/tools/read.zig — file read / MIME detection primitives
Summary
Zag can already decode PNGs (
src/png_decode.zig,src/halfblock.zig,src/buffers/image.zig) and render image-backedtool_resultnodes in the conversation tree, but users cannot attach images to their own messages. There is no.imagevariant inContentBlock,submitInputis text-only, andonUserInputSubmittedhas no image path.Motivation
Users expect to paste screenshots or diagrams into the conversation and have the model analyze them inline. Today the only workaround is describing the image in text or asking the agent to read a path manually.
Prior art
pi (
earendil-works/pi): on clipboard paste, writes the image bytes to a temp file (/tmp/pi-clipboard-<uuid>.png) and inserts the file path directly into the editor text. On submission, image paths are detected by MIME sniffing, optionally resized, converted to base64, and sent asImageContentblocks alongside the text.opencode (
anomalyco/opencode): in their web client, pasted file attachments are collected separately; in their GitHub agent, image URLs are downloaded, replaced with@filenamereferences in the prompt, and shipped as base64 attachments.Both converge on the same under-the-hood pattern: save to a temp path, insert the path into the prompt text, then detect image paths at send-time and convert them to base64 content blocks for the LLM.
Proposed behavior
.imagevariant toContentBlock(or reuse the existingImageBuffer/BufferRegistryflow) so Anthropic and OpenAI providers can serialize it correctly.Session.zigJSONL should store image references (paths or base64) so replay and resume work.Scope
ContentBlock/ provider wire serialization for imagesImageBuffer/NodeRendererplaceholder or half-block inline)Out of scope
Relevant files
src/types.zig—ContentBlockunionsrc/AgentRunner.zig—submitInputsrc/EventOrchestrator.zig—onUserInputSubmitted, paste event handlingsrc/Conversation.zig—persistUserMessage, node treesrc/providers/anthropic.zig,src/providers/openai.zig— wire serializationsrc/buffers/image.zig,src/BufferRegistry.zig— existing image buffer infrasrc/tools/read.zig— file read / MIME detection primitives