Skip to content

fix(agent-runtime): deduplicate persisted system events - #1499

Open
ZxlDragonDoctor wants to merge 1 commit into
vastsa:mainfrom
ZxlDragonDoctor:codex/fix-system-transcript-dedup-1477-pr
Open

ZxlDragonDoctor wants to merge 1 commit into
vastsa:mainfrom
ZxlDragonDoctor:codex/fix-system-transcript-dedup-1477-pr

Conversation

@ZxlDragonDoctor

Copy link
Copy Markdown
Contributor

Problem

SystemTranscriptJournal deduplicates persisted messages by object identity, so cloned or reconstructed copies of the same system event can be appended again. Transcript replay also retained identical system rows, inflating model context.

Solution

Track a normalized serialized identity across persist, restore, remember, and checkpoint paths. Collapse exact duplicate system rows during replay while preserving messages whose timestamps or tool/model state differ.

Testing

  • node packages/agent-runtime/node_modules/vitest/vitest.mjs run packages/agent-runtime/src --config packages/agent-runtime/vitest.config.ts — 94 files passed; 1303 passed, 1 skipped.
  • node 'C:\Users\朱小龙\AppData\Local\pnpm\.tools\pnpm\12.8.1\node_modules\pnpm\bin\pnpm.mjs' --filter @pi-desktop/agent-runtime typecheck — passed.
  • E2E-SYSTEM-TRANSCRIPT — NOT RUN: this Windows host has no Host Core binary or MSVC link.exe; the available MinGW target still requires the MSVC host linker. Process-level persistence across a Host restart remains unverified.

Issue alignment

Closes #1477. Repeated exact system events are collapsed; distinct timestamp or model-state events remain intact.

Track normalized system-message identities across persistence and restoration so cloned events do not inflate the session journal. Keep events with distinct timestamps or tool declarations as separate history entries.

Fixes vastsa#1477

@muzimu217 muzimu217 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified locally on macOS (worktree off current main): system-transcript journal tests 32/32 (4 files — including a test shaped exactly like the minimal repro in #1477), full agent-runtime suite 94 files / 1303 tests pass.

The fix implements the content-key direction from the issue discussion, plus the non-destructive recovery the issue asked for:

  • One canonical key (serializedSystemMessage: content text + tool declarations + timestamp) now feeds every dedup surface — persist, restore, remember, rememberCheckpoint, and isPersisted. The content-level set (serializedIds) is populated at persist/restore/remember, so the previously dead checkpoint path is alive again.
  • Semantics match the reporter's expectation precisely: identical payload with the same timestamp = the same durable event (the 256× duplicates collapse to one — covered by a clone-repro test), while the same payload at a new timestamp is a new event and is kept (dedicated test).
  • orderSystemRows now collapses exact duplicate rows during replay, so already-bloated v0.17.0 sessions recover non-destructively — rows stay on disk, context stops seeing the duplicates.

Two non-blocking observations: stored JSONL for old sessions stays bloated by design (a one-time compaction could reclaim disk, but that's optional scope beyond this fix), and rememberCheckpoint's old raw-JSON key form changed to the canonical one — no callers exist today (per the verification on #1477), so the semantic tightening is free.

fix scope, fits the current contribution window. Nothing blocking.

This branch was previously deployed

1 inactive deployment
Preview — 0aeea089 Deployed Oct 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] v0.17.0 系统消息重复持久化导致上下文膨胀、空响应及压缩失败

2 participants