Centralize jq payload preview size configuration#1066
Conversation
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Centralizes the jq middleware’s payload preview length into a single constant to remove duplicated hardcoded limits and make future adjustments consistent across code, tests, and docs.
Changes:
- Introduces
PayloadPreviewSizein the jq schema middleware and uses it for preview truncation and logging. - Updates tests to assert against
PayloadPreviewSizerather than a hardcoded value. - Refreshes middleware and agent documentation to reference the centralized preview-size limit.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| internal/middleware/jqschema.go | Adds PayloadPreviewSize and replaces hardcoded preview truncation/logging values with the constant. |
| internal/middleware/jqschema_test.go | Updates preview-length expectations and related comments to reference PayloadPreviewSize. |
| internal/middleware/README.md | Documents preview sizing in terms of PayloadPreviewSize and updates future config examples. |
| AGENTS.md | Updates agent-facing docs to reference PayloadPreviewSize for preview sizing. |
Comments suppressed due to low confidence (4)
internal/middleware/README.md:75
- Same issue as above: this line says "characters" and hardcodes 500 next to
PayloadPreviewSize, but truncation is byte-based and the constant may change. Recommend "bytes (default 500)" and dropping the explicit 500 to avoid future inconsistency.
- `payloadPreview`: First `PayloadPreviewSize` (500) characters of the JSON for quick reference
- `originalSize`: Size of the full response in bytes
internal/middleware/README.md:133
- This line references
PayloadPreviewSize = 500in docs. If the goal is to centralize configuration, consider phrasing it as "currently controlled byPayloadPreviewSize" (or "default 500") so the docs don’t become incorrect when the constant changes.
- Set custom truncation limits (currently `PayloadPreviewSize = 500`)
AGENTS.md:408
- Same mismatch here: docs say "characters" but implementation truncates by bytes. Update wording to "bytes" (or change implementation to rune-aware truncation) to avoid misleading guidance for agents when payloads include non-ASCII content.
- The `payloadPreview` shows the first `PayloadPreviewSize` characters (default 500) of the JSON for quick reference
internal/middleware/jqschema.go:350
- The inline comment says "first PayloadPreviewSize chars", but the truncation uses byte slicing on the JSON string. This can produce an invalid UTF-8 boundary and makes "chars" inaccurate. Recommend aligning terminology (bytes vs runes) and, if keeping "chars", truncate by runes instead of bytes.
// Build the transformed response: first PayloadPreviewSize chars + schema
payloadStr := string(payloadJSON)
var preview string
truncated := len(payloadStr) > PayloadPreviewSize
if truncated {
preview = payloadStr[:PayloadPreviewSize] + "..."
logger.LogInfo("payload", "Payload truncated for preview: tool=%s, queryID=%s, originalSize=%d bytes, previewSize=%d bytes",
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Verify preview truncation in Content field | ||
| preview := contentMap["payloadPreview"].(string) | ||
| assert.LessOrEqual(t, len(preview), 503, "Preview should be truncated to ~500 chars + '...'") | ||
| assert.LessOrEqual(t, len(preview), PayloadPreviewSize+3, "Preview should be truncated to PayloadPreviewSize chars + '...'") |
There was a problem hiding this comment.
This assertion and its message talk about "chars", but len(preview) is a byte count in Go. To avoid confusion (and potential issues if payloads contain non-ASCII), either update the wording to "bytes" or change the truncation logic/tests to be rune-aware.
| assert.LessOrEqual(t, len(preview), PayloadPreviewSize+3, "Preview should be truncated to PayloadPreviewSize chars + '...'") | |
| assert.LessOrEqual(t, len(preview), PayloadPreviewSize+3, "Preview should be truncated to PayloadPreviewSize bytes + '...'") |
| - **Payload Storage**: Stores complete response payloads in `/tmp/gh-awmg/tools-calls/{randomID}/payload.json` | ||
| - **Response Rewriting**: Returns a transformed response containing: | ||
| - First 500 characters of the payload (for quick preview) | ||
| - First `PayloadPreviewSize` (500) characters of the payload (for quick preview) |
There was a problem hiding this comment.
Docs describe the preview limit as "characters" and also hardcode "(500)" next to PayloadPreviewSize. Since the implementation currently truncates by bytes, and the constant value may change, consider wording like "first PayloadPreviewSize bytes (default 500)" and avoid repeating the numeric value to prevent docs drifting out of sync.
This issue also appears in the following locations of the same file:
- line 74
- line 133
| - First `PayloadPreviewSize` (500) characters of the payload (for quick preview) | |
| - First `PayloadPreviewSize` bytes of the payload (default 500, for quick preview) |
| - Payloads are organized by session ID: `{payload_dir}/{sessionID}/{queryID}/payload.json` | ||
| - This allows agents to mount their session-specific subdirectory to access full payloads | ||
| - The jq middleware returns: preview (first 500 chars), schema, payloadPath, queryID, originalSize, truncated flag | ||
| - The jq middleware returns: preview (first `PayloadPreviewSize` chars, default 500), schema, payloadPath, queryID, originalSize, truncated flag |
There was a problem hiding this comment.
This documentation refers to PayloadPreviewSize as "chars". The code currently truncates previews by bytes (and slices strings by byte index), so this should be updated to "bytes" or the implementation should be changed to be rune/character-based.
This issue also appears on line 408 of the same file.
See below for a potential fix:
- The jq middleware returns: preview (first `PayloadPreviewSize` bytes, default 500), schema, payloadPath, queryID, originalSize, truncated flag
**Understanding the payload.json File:**
- The `payload.json` file contains the **complete original response data** in valid JSON format
- You can read and parse this file directly using standard JSON parsing tools (e.g., `cat payload.json | jq .` or `JSON.parse(fs.readFileSync(path))`)
- The `payloadSchema` in the metadata response shows the **structure and types** of fields (e.g., "string", "number", "boolean", "array", "object")
- The `payloadSchema` does NOT contain the actual data values - those are only in the `payload.json` file
- The `payloadPreview` shows the first `PayloadPreviewSize` bytes (default 500) of the JSON for quick reference
| // PayloadPreviewSize is the maximum number of characters to include in the payload preview | ||
| // This controls how much of the original payload is returned inline when a payload is stored to disk |
There was a problem hiding this comment.
PayloadPreviewSize is documented as a character count, but the implementation uses len(payloadStr) and payloadStr[:PayloadPreviewSize], which operate on bytes and can also split multi-byte UTF-8 sequences. Either (a) clarify that this is a byte limit (update the comment/name), or (b) switch truncation to be rune-aware so the docs match behavior and previews don’t cut in the middle of a UTF-8 rune.
This issue also appears on line 344 of the same file.
| // PayloadPreviewSize is the maximum number of characters to include in the payload preview | |
| // This controls how much of the original payload is returned inline when a payload is stored to disk | |
| // PayloadPreviewSize is the maximum number of bytes to include in the payload preview | |
| // This controls how much of the original payload (in UTF-8 bytes) is returned inline when a payload is stored to disk |
The jq middleware payload preview size (500 characters) was hardcoded in multiple locations, making it difficult to adjust consistently.
Changes
Added
PayloadPreviewSizeconstant ininternal/middleware/jqschema.go500in preview truncation logic500in logging statementsUpdated references in documentation (
README.md,AGENTS.md) and tests to reference the constantExample
Before:
After:
The constant provides a single point of configuration for payload preview behavior across the middleware package.
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
example.com/tmp/go-build3360590955/b279/launcher.test /tmp/go-build3360590955/b279/launcher.test -test.testlogfile=/tmp/go-build3360590955/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 4047529/b162/_pkg_.a 64/src/runtime/cgo x_amd64/vet(dns block)invalid-host-that-does-not-exist-12345.com/tmp/go-build3360590955/b264/config.test /tmp/go-build3360590955/b264/config.test -test.testlogfile=/tmp/go-build3360590955/b264/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/runtime/cgo 3E2y/lfIxJsLvVk-kDynw3E2y .12/x64/bin/as(dns block)nonexistent.local/tmp/go-build3360590955/b279/launcher.test /tmp/go-build3360590955/b279/launcher.test -test.testlogfile=/tmp/go-build3360590955/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 4047529/b162/_pkg_.a 64/src/runtime/cgo x_amd64/vet(dns block)slow.example.com/tmp/go-build3360590955/b279/launcher.test /tmp/go-build3360590955/b279/launcher.test -test.testlogfile=/tmp/go-build3360590955/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 4047529/b162/_pkg_.a 64/src/runtime/cgo x_amd64/vet(dns block)this-host-does-not-exist-12345.com/tmp/go-build3360590955/b288/mcp.test /tmp/go-build3360590955/b288/mcp.test -test.testlogfile=/tmp/go-build3360590955/b288/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true se 4047529/b025/vet.cfg rgo/bin/as -p internal/msan -lang=go1.25 /opt/hostedtoolcache/go/1.25.7/x-trimpath go_.�� lcache/go/1.25.7/x64=/_/GOROOT Rd x_amd64/vet -p hash/maphash -lang=go1.25 x_amd64/vet(dns block)/tmp/go-build1404549538/b001/mcp.test /tmp/go-build1404549538/b001/mcp.test -test.testlogfile=/tmp/go-build1404549538/b001/testlog.txt -test.paniconexit0 -test.timeout=10m0s -tes�� io.containerd.runtime.v2.task/moby/63201cdb6593143a21722511515a1/run/containerd/io.containerd.rugofmt -test.timeout=10m0s k/gh-aw-mcpg/gh-aw-mcpg/awmg se 4047529/b025/vet--norc ffbfb0094a8a3294--noprofile k/gh-aw-mcpg/gh-aw-mcpg/awmg -tes�� -test.paniconexit0 y by/71af78b18fce5832ab10e180bbc3a77cfee5908e386c9517dcf943eeea458c34/log.json ntime.v2.task/mo/opt/hostedtoolcache/go/1.25.7/x64/pkg/tool/linux_amd64/vet Rd t", "REQUESTS_CA_B-c iginal(dns block)If you need me to access, download, or install something from one of these locations, you can either:
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.