fix(logs): bound complete runtime log responses

[ericcaiwx-star]

Runtime log pages could exceed the documented 512 KiB body limit because the old estimate counted raw messages before JSON escaping, cursor signing and HTTP metadata. Both readers now measure the complete serialized response and build a bounded delivered prefix. Cursor, masking and withholding state follow that raw prefix; the existing message cap, public schema, signing, authorization and legacy decoder remain.

Closes #1980.

Current main integration

Integration head 8a5bf0e9f3a1a1a2de8a9685df60f43bc66e360d normally merges upstream f1dd15739 and resolves the sole conflict in the runtime-log flow document. The resolution preserves this PR's cursor/window contract and full changelog while incorporating main's explicit-zero and positive sub-second --since rejection. The merged flow is 2,488 words, below the 2,500-word limit. No production runtime or test logic required manual conflict resolution.

Moving full-window repair

Addresses the grouped-timestamp follow report, following the earlier full-window repair. A cut inside a millisecond group now advances counted timed overlap when the observed source window is ordered and the remaining times fit it. Every wire cut also retains the existing authenticated raw-window witness, including cuts with distinct messages: an identical late copy can arrive afterwards. Changed witnesses keep delivered progress only with the existing bounded overlap context; uncertain context resets visibly.

Short, untimed, unordered, overflowing and outside-floor snapshots retain their positional baseline. Stable snapshots drain; changed uncertain windows can reset and replay. Current order cannot establish historical order. Identical saturated replacements remain unobservable, so this is not a general lossless-stream guarantee. No backend sequence or new cursor field was added. Container checkpoint loss conservatively retains one reset gap and suppresses duplicate notices; equal row counts do not establish continuity.

Executed evidence

Frozen behavior-proof head: db624383f759ebac29cb50140eba584b71bf1d72; current main-integration head: 8a5bf0e9f3a1a1a2de8a9685df60f43bc66e360d; published parent before the behavior repair: 59b9c9eec562e50df8e3cbe9669a2e6f697ba37d. Each intermediate failure and corrective review was retained, rather than reported as a pass.

Actual loopback gRPC GetSandboxLogs → repository Backend/SDK/SandboxDriver → authenticated Fastify HTTP: controlled tail 1000, about 2 KiB messages, ten distinct messages per millisecond, three appended between each of eight polls:

59b: 222,222,222,222,222,222,222,221
     1775 delivered /461 unique /1314 repeats; 560 of 1021 missed; replay 221
current: 221,221,221,221,128,3,3,3
         1021 delivered /1021 unique /0 repeats; 0 missed; replay 0

Twelve additional grouped cases (tails 200/1000, six deterministic 2–3/ms seeds) pass on the current head. One parent case was a genuine negative control; parent defects were observed in 11/12, not asserted for all seeds. A raw-identical grouped control compares the complete repeated group-ID sequence and multiplicities: current 1021 occurrences,0 excess copies,0 missing,replay 0. No unique-message metric substitutes for that check.

Further actual RPC/HTTP regressions:

incoming remembered hash after rollover:
  5da3: unread identical copy 0 (expected1), no gap
  current: unread copy 1, explicit reset gap, stable replay 0
copy arriving only after a distinct partial-group cut:
  22d: 41 of42 surviving occurrences, late copy omitted, no gap
  current: exact42 ordered occurrences, late copy 1, reset gap, replay 0
unordered original prefix becoming an ordered surviving tail:
  d5: late copy 0 (expected1)
  current: late copy 1; all originally unread rows delivered once;
           fresh100-row snapshot, explicit gap/possible replay, stable replay 0

Actual owned K3s v1.35.5+k3s1/kubelet → repository Kubernetes Driver/SDK → authenticated Fastify HTTP: deliberately seeded valid benign CRI records replace a short 80-row window with 80 newer small rows, retaining the same Pod/container and tail 100. 5da3 emits no reset gap; current returns 80 replacement rows, exactly one gap and empty replay. This is controlled CRI input, not a naturally observed rotation or timestamp-frequency claim.

Sandbox is an owned native gRPC protocol fixture peer, with fixture Compute placement; all proofs use in-memory domain state and native IAM/audit. No deployed upstream OpenShell engine, sandbox/model execution or production qualification is claimed. Artificial timestamp/order boundaries are disclosed; no host clock changed. The existing 1.8s delay control delays the original SDK call before transport.

Validation and gates

• Four affected suites on macOS: 429 total,428 passed,0 failed,1 explicit Linux MAX_ARG_STRLEN skip. Same frozen head in an owned cached official Node24 Linux container as UID 1000:429/429 passed,0 skips. The initial root-container run had a separate directory-permission skip and is retained separately.
• Existing 31 transport controls plus grouped/late-copy/rotation controls retain byte limits, full 48/16-hash and maximum-view cursor bounds, stable replay, changed values/tails, UID/restarts, PEM/withholding, large groups, untimed and delayed reads.
• TypeScript, full ESLint, affected formatting, workspace boundary, OpenAPI, docs word/site/spec checks, flow validation and diff checks pass. Flow is below 2500 words; Manual Notes are unchanged.
• Current merge head: 506 related conformance tests passed, 0 failed, 1 explicit Linux-only skip; OCC TypeScript build, workspace/module boundaries, Prettier, documentation length, go test ./internal/occcli -count=1, and go vet ./... pass. The first sandboxed attempt could not bind loopback ports; the identical unrestricted rerun passed.
• Frozen full-source independent review: scoped-clean at P0–P2. Earlier reviews found real defects, which were independently reproduced and repaired. Review itself was static; project execution is the separate evidence above.

Repair-head CI Required, Static Checks and Runtime Image Fixture succeeded. Current merge-head CI Required and Static Checks also succeed with no failed checks; optional First Agent Smoke remains in progress. ClawSweeper reviewed exact head 8a5bf0e9f3a1a1a2de8a9685df60f43bc66e360d at 5/6 with sufficient proof, no findings, Before merge None, and Ready for maintainer review. Owned server, four volumes, private kubeconfig and temporary image export were removed; RPC/HTTP listeners closed. No force push, empty commit, bypass or contributor merge.

Verification

Exact head 01092fae692ff4dd85d815fc212b21dd9734e51b. Node v26.7.0, macOS. Flow word count on this head is 2498.

• Restored the distinct OpenShell timestamp history line for OpenShell client silently normalizes impossible timestamps #2146. Checkpoint loss and a byte-cut overflow no longer each emit window_exceeded for the same Sandbox buffer loss.
• node --test --test-name-pattern "one window notice" tests/conformance/runtime-logs-sandbox.test.mjs: 1 pass, 0 fail, exit 0.
• node --test --test-name-pattern "sandbox (byte-window|follow)" tests/conformance/runtime-logs-sandbox.test.mjs: 12 pass, 0 fail, exit 0.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge.

What this changes

This PR bounds complete serialized runtime-log responses to 512 KiB and preserves continuation through byte-cut pages.

Example: An operator reads 70 container diagnostics containing many quoted arguments.

• Before: The reported Kubernetes-to-HTTP reproduction returned a 587,750-byte JSON body despite the 524,288-byte limit.
• After: The reader selects a fitting delivered prefix, includes the signed cursor and HTTP envelope in its budget, and continues remaining records with the cursor.

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	A useful, supported contract repair with production-path evidence and substantial regression coverage; all recorded findings are resolved.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (live_output): Recorded kubelet and native gRPC transport controls exercise the repository Drivers and authenticated Fastify endpoint after the behavior repair, covering bounded responses, grouped continuation, late copies and replacement recovery; subsequent changes resolve loss-notice duplication and documentation history.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Operators receive runtime-log responses larger than the documented 512 KiB limit.
Reason: The patch enforces an existing documented contract for both sources without adding configuration, permissions or a public API.

Merge readiness

✅ Ready for maintainer review

This PR fixes a documented limit that current main still exceeds. Both prior P3 findings are resolved, and no actionable defect remains in the reviewed head.

Priority: P2
Reviewed head: 01092fae692ff4dd85d815fc212b21dd9734e51b

Before merge

None.

Findings

None.

Agent review details

How this fits together

OCC runtime-log readers receive authorized revision-scoped Driver output, sanitize it, and return bounded records and authenticated continuation cursors through the HTTP API.

flowchart LR
  A[Console or CLI] --> B[HTTP runtime logs route]
  B --> C[OCC authorization and audit]
  C --> D[Compute or Sandbox Driver]
  D --> E[Sanitization and continuation]
  E --> F[Serialized response budget]
  F --> G[Records and signed cursor]

Loading

Technical review

Best possible solution:

Land the shared serialized-page budget with its authenticated prefix continuation and conservative masking and loss reporting.

Do we have a high-confidence way to reproduce the issue?

Current-main source confirms the incomplete size estimate, and the linked report records a real 587,750-byte HTTP response; this review did not execute target code.

Is this the best way to solve the issue?

Measuring the complete sanitized response in the existing shared reader boundary fixes the contract directly, while prefix-derived continuation avoids dropping records after serialization.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against d2fe579cd51f.

Provenance checked

• Container and Sandbox page limits changes intended behavior with a stated reason (feat(agents): runtime status and container log reads with console logs tab #696: Runtime logs were intentionally bounded and sanitized for operator diagnostics; the initial raw-text estimate did not account for complete JSON responses.)
• Sandbox overlap and loss detection keeps the original intent (feat(openshell): sandbox log source #726: Timestamp overlap preserves delayed and repeated occurrences within bounded history; loss notices disclose ring-buffer uncertainty.)
• PEM masking across pages keeps the original intent (fix(runtime-logs): preserve PEM context across polling pages #759: Only delivered, reliably ordered chronology can close carried PEM context; ambiguous boundaries remain masked.)
• Container frontier counts keeps the original intent (fix(logs): count delivered lines at the cursor timestamp #1918: Authenticated counts and matching suffix hashes prevent replay of timestamp groups larger than the retained hash history.)

Testing

Proof path: shipped entry point.

Security

None.

Evidence

What I checked:

• Current main still needs the fix: Current main estimates response size from raw messages capped at 8 KiB plus 256 bytes per line; it does not measure JSON escaping, cursor or envelope. The linked report supplies actual Kubernetes/SDK/authenticated HTTP reproduction: Runtime log responses exceed the documented 512 KiB wire limit #1980. (packages/occ/src/runtime-logs/read.ts:397, d2fe579cd51f)
• Complete response accounting: The shared budget measures serialized pages and the fixed server-generated request-ID envelope; the actual route retains those page fields and disables caller-supplied request IDs. (packages/occ/src/runtime-logs/page-budget.ts:16, 01092fae692f)
• Previous findings resolved: The latest commit suppresses overlapping Sandbox loss notices, adds the reported ordered-replacement HTTP regression, and restores the distinct OpenShell timestamp history entry. The earlier moving-window reports by freeqaz also informed the retained recovery coverage: fix(logs): bound complete runtime log responses #1981 (comment). (packages/occ/src/runtime-logs/sandbox.ts:335, 01092fae692f)
• Recorded production-path proof: The full PR body and contributor comments record actual kubelet/Driver/SDK/authenticated HTTP controls and native gRPC peer/Backend/SDK/SandboxDriver/HTTP controls. Grouped moving-window proof delivers 1,021 occurrences without excess or missing copies; fixture and deployment limits are explicitly disclosed. (db624383f759)
• Exact-head required CI passed: CI Required completed successfully for the reviewed head: https://github.com/openclaw/openclaw-enterprise/actions/runs/38167737273/job/114554362496. (01092fae692f)
• Authorization and cursor boundary retained: Every page request passes current exact-Agent authorization before Driver reads; cursor signing, principal/Agent/revision/source binding and expiry remain enforced. Existing route tests cover foreign cursors and revoked grants with no additional Driver reads. (packages/occ/src/index.ts:2830, 01092fae692f)

Review metrics

Metric	Value	Why it matters
Introduced scope	Production +787/-278; tests +2186/-11; documentation +118/-82	Most growth protects serialized bounds and continuation across repeated timestamps, changing windows, untimed rows, identity resets and masking; the continuation state accounts for the production growth.

Labels

Label changes:

• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
• remove rating: 🦐 gold shrimp: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
• remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

• P2: This repairs a reproducible runtime-log response-limit violation with a bounded operator-facing scope.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
• proof: sufficient: Contributor real behavior proof is sufficient.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (16 earlier review cycles; latest 8 shown)

• reviewed 2026-10-10T04:31:58.164Z sha 59b9c9e :: needs maintainer review before merge. :: none
• reviewed 2026-10-10T05:15:09.696Z sha 59b9c9e :: needs changes before merge. :: [P1] [P1] Preserve delivered overlap when a moving Sandbox cut splits a timestamp group | [P3] [P3] Suppress false and duplicate Container window-loss notices
• reviewed 2026-10-10T06:56:24.093Z sha db62438 :: needs changes before merge. :: none
• reviewed 2026-10-10T07:04:07.691Z sha db62438 :: needs maintainer review before merge. :: none
• reviewed 2026-10-11T04:21:22.013Z sha 8a5bf0e :: needs maintainer review before merge. :: none
• reviewed 2026-10-11T04:26:46.943Z sha 8a5bf0e :: needs maintainer review before merge. :: none
• reviewed 2026-10-11T15:25:07.513Z sha c31a464 :: needs changes before merge. :: [P3] [P3] Trim the runtime-log flow below the documentation hard limit | [P3] [P3] Suppress a second Sandbox notice for the same buffer loss
• reviewed 2026-10-11T15:58:06.163Z sha 59aafc1 :: needs changes before merge. :: [P3] Suppress a second Sandbox notice for the same buffer loss | [P3] Restore the distinct OpenShell timestamp history entry

Reviewed October 11, 2026, 3:27 PM ET / 19:27 UTC (Revision 17).

[ericcaiwx-star]

Fixed in a823952. Compact hash strings and window tuples retain all occurrences and identities; Sandbox checkpoints reuse their outer baseline, and container checkpoints keep their distinct baseline. Released array-hash cursors still decode. Admission limits are unchanged.

The published parent returns HTTP 500 on the full-48-anchor native SDK/HTTP cut. This head delivers all 150 new rows in 55/55/40 with empty replay. Maximum accepted Driver identity projection drains 54/54/42 with cursor/payload 1,905/1,858; conservative full-state maxima are 1,930/1,883. Related checks: 399 pass, one Linux-only skip. Real owned kubelet and protocol/SDK/HTTP controls were rerun; Sandbox peer and maximum-name projection are explicit fixtures. Temporary owned resources are removed.

[ericcaiwx-star]

Exact-head CI for a823952bf481d246e38b5b70aa6e3b81237b313e is blocked: CI Required fails because Runtime Image Fixture stops during base-build setup, before its application tests. Its receipt records pending/unknown; the aggregate log does not expose the underlying exception, so its exact cause is not established.

Main c0a882ce independently has the same pre-test fixture/unknown receipt. The separate smoke jobs on this head and main explicitly report Docker Hub anonymous pull limits. Smoke is advisory; the required fixture remains unwaived. This account cannot rerun upstream jobs. Keeping Draft until maintainer CI recovery/retry; no source change or empty commit is warranted by these setup results.

[ericcaiwx-star]

@clawsweeper re-review

[clawsweeper]

🦞👀
Exact review queued.

Re-review progress:

• State: Complete
• Detail: A newer review tuple already exists; this stale result was superseded.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/38111410518
• Updated: 2026-10-11T04:23:24.052Z