Skip to content

feat(cli): export a full Session transcript as Markdown from persistence - #6036

Open
ggbdpq wants to merge 3 commits into
apache:mainfrom
ggbdpq:fix/5958-transcript-export-markdown
Open

ggbdpq wants to merge 3 commits into
apache:mainfrom
ggbdpq:fix/5958-transcript-export-markdown

Conversation

@ggbdpq

@ggbdpq ggbdpq commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #5958. The three surfaces each expose a partial slice of a Session transcript: the Desktop export renders only what the viewport has loaded (long sessions truncate silently), the TUI copy is clipboard-bound, and the bundle exporter writes a machine-oriented archive. This adds the missing read-side path: persistence to one complete, human-readable Markdown file.

maka session-export-markdown --workspace-root <dir> --session <id> --out <file.md> [--include-thinking]

It reads every stored row through the session store's snapshot reader and renders the Desktop export's shape (## You / ### Tool calls / ## Maka) with the differences the issue asks for:

  • tool results are included (each call carries its result), since they are exactly what a debugging reader needs; every string that reaches the file passes redactSecrets;
  • thinking stays out by default, following the @kenji precedent recorded on the Desktop export, with --include-thinking as the explicit opt-in;
  • operational rows stay out (token usage, turn state, permission decisions, system notes, coordination) — they are not something anyone said.

No wire, protocol, or storage-schema change. On the issue's open questions: the thinking/tool-result boundary above is my proposed default and is easy to flip; formatter ownership is deliberately untouched — the Desktop renderer stays where it is, and converging the two implementations into a shared package can follow once that question is decided.

Verification

Check Result
packages/runtime new suite (session-transcript-export.test.ts, 6 tests: full fidelity, redaction, thinking flag, operational rows, structured failures, pure renderer) 6/6 pass (red before the implementation)
packages/cli new suite (session-export-markdown-command.test.ts, 4 tests: happy path, never-overwrite, missing session exit 2, missing args exit 1) 4/4 pass
Adjacent session-export.test.ts 25/25 pass
biome check on all touched files clean
npm run check:asf-headers clean
tsc -b runtime / tsc --noEmit cli clean; cli reports one pre-existing @maka/eval TS2307 on this machine (module dist not built locally), reproducible on main without this change

Not run locally: the full packages/cli test suite has three compiled* tests that depend on a built launcher; they fail identically on main without this change (verified by stashing) and are exercised by CI on the PR head.

AI use

Implemented with GLM-5.3-Flash (ZCode); human-reviewed.

Checklist

  • Tests cover the new behavior and fail without the change
  • biome check and check:asf-headers pass on touched files
  • No schema, wire, or protocol change

The three surfaces each expose a partial slice of a Session transcript:
the Desktop export renders only what the viewport has loaded (long
sessions truncate silently), the TUI copy is clipboard-bound, and the
bundle exporter writes a machine-oriented archive. apache#5958 asks for the
missing read-side path: persistence -> one complete, human-readable
Markdown file.

Add `maka session-export-markdown --workspace-root --session --out
[--include-thinking]`. It reads every stored row through the session
store's snapshot reader and renders the Desktop export's shape (`## You`
/ `### Tool calls` / `## Maka`) with the differences the issue asks for:

- tool results are included (each call carries its redacted result),
  since they are exactly what a debugging reader needs; every string
  passes `redactSecrets`;
- thinking stays out by default per the @kenji precedent on the Desktop
  export, with `--include-thinking` as the explicit opt-in;
- operational rows (token usage, turn state, permission decisions,
  system notes, coordination) stay out.

No wire, protocol, or storage-schema change; the Desktop renderer is
left in place, so formatter ownership (the issue's open question) is
untouched by this.

Fixes apache#5958

Generated-by: GLM-5.3-Flash (ZCode)
@github-actions github-actions Bot added the effort/L Under 1000 readable lines label Oct 9, 2026

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

At a8affc4e2feccfa97504696b1bd3e99dd3be5de9, I found four reproducible P2 issues in the new Markdown export. The command adds a separate Runtime reader and formatter, includes tool results, and makes thinking opt-in; it does not change the existing Desktop exporter. There were no previous reviews to resolve.

  • [P2] Read the current transcript authority — packages/runtime/src/session-transcript-export.ts:83. This reads legacy session_messages, whereas current conversations are projected from runtime_events. In real execution stores, a completed invocation projected five persisted messages, but this exporter returned success with messageCount: 0 and only a title/timestamp. The existing host transcript reader already reads the event ledger. This is not an unflushed or partially completed turn.
  • [P2] Enforce no-overwrite at publication — packages/cli/src/session-export-markdown-command.ts:91. An output created after the existence check was truncated and the command returned 0. An existing dangling symlink also passed the check and caused its target to be created. The initial check does not replace exclusive creation or atomic no-replace publication.
  • [P2] Redact every rendered tool-result branch — packages/runtime/src/session-transcript-export.ts:216,220. A terminal command is inserted into its label without redaction, and valid shell_run results use the unredacted JSON fallback. Recognizable fixture credentials survived both paths, while terminal output was correctly redacted. File-path labels need the same treatment. This finding concerns tool results, not the existing Desktop choice to preserve user-authored text.
  • [P2] Preserve mid-turn user messages — packages/runtime/src/session-transcript-export.ts:135. Selecting the first user row silently drops later steering messages in the same turn. Both a real ledger projection and valid persisted message rows reproduced this. Changing the storage reader alone would not fix the formatter.

Verification: Core, Storage, MCP, Runtime, Runtime Host, Eval, and CLI built successfully on Node 22.23.2. The six new Runtime tests, four new CLI tests, 25 adjacent bundle-export tests, and 107 CLI/read-model/host-transcript tests all passed: 142 tests, no skips. Scoped Biome and git diff --check passed. The additional probes used isolated SQLite stores and fabricated credentials; no actual user data or live provider was used. For the overwrite race, only the stat timing was controlled to create a second writer's file before the real export/write proceeded; the dangling-symlink reproduction required no interception.

Large sessions and fidelity limits: A synthetic legacy-table session with 1,200 turns/4,800 rows exported its first and last turns into 5,065,610 bytes in approximately 170 ms; observed RSS and heap-used increases during export were about 27.6 MiB and 23.7 MiB. This is a finite Linux sample, not an upper-bound guarantee: all source rows, grouping maps, output lines, and the final string are retained rather than streamed. Tool arguments and attachment details are not rendered, archived results remain placeholders, and grouping tools before all assistant text does not preserve within-turn chronology. Thinking is excluded by default and redacted when enabled; operational rows are omitted.

Compatibility and merge state: There is no protocol, epoch, or storage-schema change. At the same head, hosted checks showed 13 successes and one skipped installed-CLI Eval check; CI and CLI package validation passed. The branch merges without conflicts with main 66da38efd33475c979233c41bf98f9b8855a3dff; GitHub reports MERGEABLE/BLOCKED, so there is no source conflict, but that is not merge approval. The full monorepo suite, packaged CLI installation, macOS/Windows execution, and Desktop end-to-end tests were not rerun locally. The four correctness issues need addressing; the feature's default content policy and merge decision remain for the maintainers.

Comment thread packages/runtime/src/session-transcript-export.ts Outdated
Comment thread packages/cli/src/session-export-markdown-command.ts Outdated
Comment thread packages/runtime/src/session-transcript-export.ts Outdated
Comment thread packages/runtime/src/session-transcript-export.ts Outdated
…omically

Review fixes for the four reproducible P2s on apache#6036:

- Read messages through the runtime event ledger read model; sessions
  older than the ledger fall back to their stored rows. A ledger-backed
  session no longer exports as a successful empty transcript.
- Render every user message in a turn, so mid-turn steering corrections
  stop disappearing while the responses to them remain.
- Redact terminal labels and the structured-result fallback; shell_run
  fields and terminal command lines no longer serialize verbatim into
  the export.
- Create the destination exclusively after the friendly stat precheck:
  a path created between the check and the write, or a dangling symlink
  the check reads as absent, now reports destination_exists (exit 5)
  instead of being truncated or followed.

Generated-by: GLM-5.3-Flash (ZCode)

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

At 842f3dada065e87dafc6015d450fd6efe5cc0b75, three earlier P2 issues are fixed and the tool-result redaction issue is only partially fixed; the remaining gap is graded P3.

Earlier issue Current result
Reading only legacy session_messages Fixed: a real completed invocation projects and exports all five messages, including both answers and the steering input.
Overwriting a competing file or following a dangling symlink Fixed: exclusive wx creation returns exit 5; the competing file survives and the symlink target is not created.
Unredacted structured results and labels Partially fixed: terminal commands/output and the structured fallback are redacted, but file-result paths still bypass redaction.
Dropping mid-turn user messages Fixed: both opening input and steering input appear in the output.

Remaining P3 — redact file-result paths too. In packages/runtime/src/session-transcript-export.ts:242–247, file_diff.paths is interpolated into the label and file_write.path into the body without redactSecrets. Valid results written through the real Session store still export the synthetic secret-shaped markers sk-path-fixture123 and sk-write-fixture123 unchanged, although redactSecrets replaces both and the diff-body control is redacted. Apply redaction to these branches as well, or centrally to every tool-result label/body, and add regression tests for both result types.

Severity note: this is graded P3 rather than P2 because it needs a secret-shaped string inside a file path. That is reproducible with a fixture, but far less likely in practice than the command and output credentials fixed in this revision. It is still worth closing for consistency.

No additional issue was established in the new reader/publication changes. The existing role-grouped output still does not preserve interleaved narration order; this is not a new finding. The Runtime Host protocol and storage schema are unchanged.

Dependency/CLI builds, related tests and the real-store/competition probes pass locally. Hosted checks are green except the skipped CLI Eval lane; GitHub reports mergeable. This does not establish a human merge decision. The full monorepo suite, Windows/macOS probes and installed-package execution were not rerun.

Comment thread packages/runtime/src/session-transcript-export.ts Outdated
@github-actions github-actions Bot added effort/XL Under 2500 readable lines and removed effort/L Under 1000 readable lines labels Oct 10, 2026
Review follow-up on apache#6036 (Astro-Han P3): the file_diff label carried
`content.paths` and the file_write line carried `content.path` without
passing through `redactSecrets`, so a secret baked into a path or
filename reached the export verbatim. Both labels now go through the
same redactor as the diff and write bodies.

Generated-by: GLM-5.3-Flash (ZCode)

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

At fb90098aa6c9206e8f9fd537875cd696df47280f, the remaining P3 file-path redaction finding is fixed. The file_diff label and file_write destination now pass through redactSecrets (packages/runtime/src/session-transcript-export.ts:242–255). An independent real-Session-store probe confirms that both synthetic secret-shaped paths and the diff-body marker are absent from the export, while an ordinary filename and the write byte count survive. The same probe fails on the previous reviewed head 842f3dad and passes again after restoring this head.

The increment changes only those two rendering branches and their regression tests; no rebase or unrelated implementation change was introduced. The earlier ledger-reader, steering-message and exclusive-destination fixes are unchanged, and the complete transcript/Bundle-export and CLI Markdown-command suites pass all 40 tests, including existing-file and dangling-symlink refusal. No additional P0–P3 issue was established in this increment. The role-grouped output still has the previously documented chronology limitation; this revision does not claim to change it. User-authored text and Session titles retain their existing treatment.

I read the current diff and compared it with the prior reviewed head, checked all result branches and their added assertions, and rebuilt the Runtime, Runtime Host, Eval and CLI against this batch's fresh dependency installation and unchanged dependency inputs. Biome checks and git diff --check pass. Protocol epoch and storage schema are unchanged. GitHub reports this exact head mergeable, with successful test, audit, package/platform checks and a skipped CLI Eval lane; merge-tree against fetched main 16a8f5b8 is clean.

This validates persisted fixtures and the local command implementation, not a full-monorepo run, live provider execution, or independent Windows/macOS testing. Feature adoption and merge remain human decisions.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XL Under 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: full-fidelity session transcript export from persistence (Markdown)

2 participants