Session forking writes full parent rollout into child JSONL (storage amplification despite forked_from_id) #35647

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   CLIIssues related to the Codex CLIIssues related to the Codex CLI

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   sessionIssues involving session (thread) management, resuming, forking, naming, archivingIssues involving session (thread) management, resuming, forking, naming, archiving

   performance

   on Jul 27, 2026
2. github-actions commented on Jul 27, 2026

   

   github-actionsbot

   on Jul 27, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • Codex Desktop: screenshots re-persisted in full on every compaction and inherited by subagent forks - ~/.codex/sessions reached ~165 GiB (95% base64 images) #35458

   Powered by Codex Action
3. Yasei-no-otoko commented on Jul 28, 2026

   

   Yasei-no-otoko

   on Jul 28, 2026 – with ChatGPT Codex Connector

   Author

   More actions

   Implementation update (2026-07-28):

   • Full open-PR audit: 193 open PRs checked; no existing PR implements managed history_base references plus fork-local JSONL deltas. [core] Drop parent MCP lifecycle events from forked agent history #30977 is related payload filtering but does not change child persistence.
   • Fix branch: Yasei-no-otoko:session-jsonl-amplification-upstream
   • Head: 61cb3d7565bf6409ea10bd55082675797086085b
   • Updated onto upstream main at 49025589b0216b876b1a6a20977536c7d55cdb8b via merge commit 477a26d4d2.
   • Validation: cargo +1.95.0 test -p codex-thread-store --lib => 192 passed, 0 failed; package fmt check passed.
   • Added regression coverage for concurrent archive versus legacy fork mode dispatch and Paginated Latest cutoff after blank/rejected complete physical tail lines.
   • The two reported local child rollouts are now delta-only references: 67,930 bytes / 16 records and 23,463 bytes / 2 records, each with a frozen parent history_base cutoff.

   Compare URL: https://github.com/openai/codex/compare/main...Yasei-no-otoko:codex:session-jsonl-amplification-upstream?expand=1

   I cannot open the upstream PR from the current account: GitHub returns Yasei-no-otoko does not have the correct permissions to execute CreatePullRequest because this repository's external contribution flow is invitation-only. The branch is pushed and ready for an invited maintainer/account to open the PR. The PR should use Fixes #35647 and Related to #22593.
4. Yasei-no-otoko commented on Jul 28, 2026

   

   Yasei-no-otoko

   on Jul 28, 2026

   Last edited by Yasei-no-otoko

   Author

   More actions

   Additional reproduction details for Issue 1 have been confirmed.

   The affected child’s legacy history_base.end_byte_offset falls immediately after a newline in the parent JSONL, so it is physically at a complete record boundary. However, the preceding record’s token_count.rate_limits used an older or newer schema that differs from the current Rust type. As a result, legacy_fork::last_complete_jsonl_offset_at_or_before treated a nested RolloutLine deserialization failure as incomplete JSONL, causing resume to stop with:

   invalid rollout history lineage for 019fa108-23a7-7921-ab39-0092e1227f2d: cutoff byte offset is not at a complete JSONL record
   

   The fix restricts cutoff validation to the stable outer rollout envelope (timestamp: string, type: string, payload: object) and permits evolution of nested payload schemas. It requires the rollout-envelope shape rather than accepting arbitrary objects, and continues to reject malformed envelopes and payload: null or array values.

   The regression tests and all 195 codex-thread-store tests pass.

   Fix PR (submitted from a fork): Yasei-no-otoko#5
5. MengShengbo commented on Jul 31, 2026

   

   MengShengbo

   on Jul 31, 2026

   More actions

   I reproduced this on Codex CLI 0.146.0 on Linux x86_64 and have a second, sanitized data point that narrows the high-frequency trigger.

   Observed chain

   A single long-lived process launched as plain codex created seven top-level user-thread children over 54 minutes. Their rollout sizes were:

   62.68 MiB
   63.56 MiB
   63.80 MiB
   63.87 MiB
   63.98 MiB
   64.02 MiB
   65.08 MiB
   

   All seven canonical session_meta records reported:

   source = cli
   originator = codex-tui
   thread_source = user
   history_mode = legacy
   cli_version = 0.146.0
   

   Each child had a fresh thread ID and forked_from_id pointing to the immediately preceding thread, producing one continuous lineage. This is about 447 MiB of child rollouts from one active conversation in under an hour.

   Trigger classification

   This chain is not byte-identical duplicate files and does not look like same-ID resume corruption. It is source-preserving prompt-edit/turn-replacement branching:

   • Several source turns ended with the durable <turn_aborted> marker.
   • For four corrected prompts, the discarded prompt occurred twice in the source rollout (response_item + event_msg) and zero times in the child's inherited initial batch.
   • The replacement child retained the earlier conversation and excluded the interrupted prompt, which is consistent with a branch-before-turn operation.
   • Model and effort remained gpt-5.5 / medium; I found no persisted model-safety-buffering event in the chain, so this does not match the faster-model safety retry path.

   The branch behavior itself appears intentional. The storage amplification is not: prompt editing is a normal, potentially frequent TUI workflow, and every edit materialized nearly the complete legacy parent history again.

   Persistence evidence

   On each child creation, inherited rollout items were serialized again with timestamps clustered within roughly one second of the child's creation time. Historical session_meta records were inherited too, so the number of persisted session_meta lines increased by one at each generation. This explains why exact-file hashing finds no duplicates even though almost all logical history is copied.

   The current implementation trace matches the analysis already posted here and in #22593:

   • legacy root/user forks reach ForkPersistence::Copied;
   • record_initial_history persists the reconstructed inherited items into the child rollout;
   • paginated forks already have the reference-backed history_base path from Support paginated thread forks #35220, but existing and newly continued legacy threads remain on copied persistence.

   I am also aware of the external patch at Yasei-no-otoko#5 and do not want to create a competing unsolicited implementation.

   Contribution request

   The repository contribution guide requires an explicit maintainer invitation before opening an external PR. If the team wants an external contribution here, I can prepare a focused, test-first PR against the preferred architecture rather than submitting another broad lineage rewrite.

   A possible first reviewable stage would be a regression test for a legacy TUI prompt-edit fork showing that the child persists a bounded parent reference/delta rather than reserializing the complete parent rollout. Please confirm whether the intended direction is:

   1. reference-backed legacy forks,
   2. migration/new-thread defaulting to paginated history, or
   3. another staged approach already planned internally.

   I can provide additional aggregate-only measurements without exposing conversation contents.
6. boombx403-byte commented on Aug 18, 2026

   

   boombx403-byte

   on Aug 18, 2026

   More actions

   Hi @Yasei-no-otoko, this subagent persistence/history issue aligns with some boundary anomalies observed in multi-agent rollouts. Codex Rescue Alpha5 provides read-only lifecycle and subagent boundary diagnostics, cleanly separating historical start markers from current live execution state without altering the source rollout.

   If you have access to the local session, you can run a non-destructive check:

   pip install codex-rescue==0.1.0a5
   codex-rescue doctor --latest

   No raw session data is required, and please redact private paths if you share any output.
7. EthanSK commented on Aug 31, 2026

   

   EthanSK

   on Aug 31, 2026

   More actions

   Additional macOS Desktop reproduction with an exact large-fork measurement:

   • Codex Desktop 26.825.51511 (build 7377) on macOS arm64 created a new child with forked_from_id pointing to its long-running parent.
   • The child's first feedback upload failed with HTTP 413 Request Entity Too Large only 101 seconds after the child was created.
   • Streaming both append-only rollouts only through that failure timestamp found 829,661,473 bytes (791.2 MiB), 100,070 records in the child versus 831,318,777 bytes, 100,175 records in the parent.
   • The 101-second-old child was therefore already 99.8% of the parent's byte size. It contained child and inherited parent session_meta records plus the inherited history.

   This also creates a concrete downstream failure beyond disk use: Desktop's Help > Send Feedback defaults session logs on, attaches the physically copied child rollout, and rejects the report before it can be sent. That uploader symptom is tracked in #40822.

   A truly independent blank task is a separate case. This reproduction shows a new-looking fork becoming huge immediately; it does not indicate a stale cache or selection of an unrelated pinned task.

   No private task IDs, report text, transcript content, logs, screenshots, local paths, or extra files are included here.