Skip to content

perf(web-shell): keep streaming output responsive - #9405

Merged
ytahdn merged 2 commits into
QwenLM:mainfrom
ytahdn:worktree-web-shell-stream-perf
Aug 19, 2026
Merged

perf(web-shell): keep streaming output responsive#9405
ytahdn merged 2 commits into
QwenLM:mainfrom
ytahdn:worktree-web-shell-stream-perf

Conversation

@ytahdn

@ytahdn ytahdn commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

Reduces Web Shell main-thread contention during long thinking and assistant streams by batching transcript delivery, prioritizing composer input, preserving stable history identities, limiting repeated tail derivations, and avoiding expensive Markdown parsing while very large content is still growing. Adds a deterministic browser performance scenario so the behavior can be reproduced without relying on model output.

Why it's needed

Long retained transcripts caused each stream delta to repeat whole-history projection, grouping, virtualization, and Markdown work, which competed with editor input. Under the fixed 5,000-turn and 400-chunk workload, current main measured a median 17.8s stream phase, 600ms typing overhead, and 92 long tasks; the optimized path measured about 8.0s, 327ms, and 1 long task across three runs.

Reviewer Test Plan

How to verify

Open a Web Shell session with a long transcript and stream a large thinking or assistant response while continuously typing in the composer. Verify typed characters are not dropped, the composer remains responsive, the final response sentinel appears, short streams keep live Markdown rendering, large streams restore full Markdown after settling, compact mode updates the active tail, and resetting a session never displays discarded transcript blocks. The opt-in deterministic performance scenario should complete with all correctness assertions and print timing metrics for comparison.

Evidence (Before & After)

The deterministic workload uses 5,000 retained turns, 400 chunks, and a 10ms chunk interval.

Measurement main median optimized median
Stream phase 17,839ms 7,993ms
Typing overhead 600ms 327ms
Long-task count 92 1

The final post-rebase verification completed in 6,950ms with 291ms typing overhead and one 56ms long task. These figures compare practical versions rather than a strict isolated ablation because the original worktree baseline and latest main were not identical commits.

Tested on

Platform Result
macOS Passed
Windows Not tested
Linux Not tested

Environment (optional)

macOS, Node.js 22.14.0, Chromium via Playwright.

Risk & Scope

  • Main risk or tradeoff: transcript rendering is intentionally paced during streaming, and very large growing Markdown is temporarily shown as escaped plain text before full formatting is restored when the block settles.
  • Not validated / out of scope: manual performance testing on Windows and Linux, and timing claims for nondeterministic real-model output.
  • Breaking changes / migration notes: None.

Linked Issues

N/A

中文说明

本 PR 做了什么

通过批量投递 transcript、优先保障输入框交互、维持历史消息对象身份稳定、减少尾部重复派生,并在超大内容持续增长时避免昂贵的 Markdown 重复解析,降低 Web Shell 在长 thinking 和 assistant 流式输出期间的主线程竞争。同时增加确定性的浏览器性能场景,使测试不再依赖模型输出的随机性。

为什么需要

当 transcript 保留大量历史记录时,每个流式增量都会重复执行整段历史的投影、分组、虚拟列表和 Markdown 工作,与编辑器输入争抢主线程。在固定的 5000 轮历史、400 个 chunk 负载下,当前 main 的中位结果为流式阶段 17.8 秒、输入额外开销 600 毫秒、92 个长任务;优化方案三次运行的中位结果约为 8.0 秒、327 毫秒和 1 个长任务。

Reviewer 测试计划

如何验证

打开包含很长 transcript 的 Web Shell 会话,在持续输出大型 thinking 或 assistant 内容时连续在输入框中输入。确认字符不会丢失、输入框保持响应、最终响应标记可见、短流仍实时渲染 Markdown、大型流结束后恢复完整 Markdown、紧凑模式持续更新活动尾部,并且同一会话重置后不会显示已丢弃的 transcript block。可选的确定性性能场景应通过全部正确性断言并输出可比较的时序指标。

证据(前后对比)

确定性负载固定为 5000 轮历史、400 个 chunk、每个 chunk 间隔 10 毫秒。

指标 main 中位数 优化后中位数
流式阶段 17,839ms 7,993ms
输入额外开销 600ms 327ms
长任务数量 92 1

最终 rebase 后的一次验证结果为流式阶段 6,950 毫秒、输入额外开销 291 毫秒、1 个 56 毫秒长任务。由于最初 worktree 的基线与最新 main 并非完全相同的提交,上述数据是实际版本对比,不是严格的单变量消融实验。

测试平台

平台 结果
macOS 通过
Windows 未测试
Linux 未测试

环境(可选)

macOS、Node.js 22.14.0、Playwright Chromium。

风险与范围

  • 主要风险或取舍:流式期间 transcript 会按固定节奏渲染;超大且仍在增长的 Markdown 会暂时以转义后的纯文本显示,并在内容稳定后恢复完整格式。
  • 未验证或不在范围内:Windows、Linux 的手动性能测试,以及非确定性真实模型输出的具体时序指标。
  • 破坏性变更或迁移说明:无。

关联 Issue

无。

@ytahdn

ytahdn commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

E2E performance report

The deterministic Chromium scenario passed after rebasing onto the latest main.

  • Workload: 5,000 retained turns, 400 streaming chunks, 10ms chunk interval
  • Replay: 889ms
  • Stream: 6,950ms
  • Typing duration: 1,291ms
  • Typing overhead: 291ms
  • Long tasks: 1 (56ms total / 56ms max)
  • Slow input events: 133 (24ms p95 / 32ms max)
  • Correctness: prompt submission, concurrent typing, streamed completion, and final sentinel assertions passed

Related verification also passed: SDK and Web Shell production builds, SDK/Web Shell/WebUI type checks, 419 Web Shell tests, 301 SDK transcript tests, and 217 WebUI session-provider tests.

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR — this is a well-instrumented one.

Template looks good ✓

Problem: observed and measured, not theoretical. The design doc carries browser profiling (the 104 ms long task dominated by applyTurnCollapse and repeated full-history derivations), and the PR commits a deterministic reproduction — the opt-in WEB_SHELL_PERF=1 Playwright scenario replaying 5,000 turns + 400 chunks. Reported before/after on macOS: stream phase 17.8 s → 8.0 s, typing overhead 600 → 327 ms, long tasks 92 → 1. Those numbers are the author's own measurements (macOS only) — noted as a claim at this stage, not independently re-run.

Direction: aligned. Composer responsiveness during long streams is a real user-facing problem for Web Shell, and the area is actively invested in — the reference product's changelog carries a directly analogous item ("Improved fullscreen streaming: long sessions stay responsive because the whole conversation is no longer re-normalized on every update").

Size: this is a cross-package change (sdk-typescript daemon/ui surface + webui DaemonSessionProvider + web-shell), which puts it under the core-infrastructure definition. Production logic is ~723 lines vs ~856 test lines vs a 75-line design doc (1,654 churned lines total). For a perf PR that's not a hard block, but 500+ production lines on a cross-package protocol surface is flagged for maintainer awareness — the blockIndexById Readonly/COW contract and the dispatch-batch timing (0 → 16 ms) are consumed beyond web-shell, so a maintainer should confirm the downstream surface is covered.

Approach: the scope feels justified rather than padded — each mechanism maps to a measured hotspot (dispatch batching, 50 ms render throttle with an input-quiet window, deferred snapshots, COW index + WeakMap-normalized tool content, timer keyed on presence, bounded Markdown parsing, tail-only fast paths), and the design doc names explicit non-goals (no incremental projector, no worker, no daemon event-ordering changes). Two things I'll look at closely in code review: the unexplained '"insight_' string guard in the projection fast path, and the set of hand-rolled ref caches in MessageList whose key checks and memo dependency lists must stay in sync by hand.

Risk: no elevated risk signals — none of the changed files match the revert-correlated high-risk paths.

Moving on to code review. 🔍 Heads-up: because Stage 0 flagged this for maintainer awareness, the final decision here caps at a defer to maintainer rather than an auto-approve, regardless of how the review goes.

中文说明

感谢贡献!这个 PR 的证据非常充分。

模板完整 ✓

问题:已观测且有实测数据,不是理论性问题。设计文档附有浏览器 profiling(104 ms 长任务主要由 applyTurnCollapse 和重复的全量历史派生构成),并且 PR 自带确定性复现——WEB_SHELL_PERF=1 的可选 Playwright 场景(回放 5000 轮历史 + 400 个 chunk)。作者报告的 macOS 前后对比:流式阶段 17.8 s → 8.0 s,输入额外开销 600 → 327 ms,长任务 92 → 1。这些数字是作者自测(仅 macOS)——此阶段按"作者声明"记录,未独立复现。

方向:对齐。长流式输出期间输入框的响应性是 Web Shell 真实的用户体验问题,且该方向有持续投入——参考产品的 changelog 有直接类似的条目("Improved fullscreen streaming: long sessions stay responsive because the whole conversation is no longer re-normalized on every update")。

规模:这是跨包改动(sdk-typescript daemon/ui 面 + webui DaemonSessionProvider + web-shell),属于核心基础设施定义范围。生产逻辑约 723 行、测试约 856 行、设计文档 75 行(总变更 1654 行)。对 perf 类 PR 不构成硬性拦截,但跨包协议面上 500+ 生产行需要提请维护者关注——blockIndexById 的 Readonly/COW 契约和派发批处理时序(0 → 16 ms)的消费方不止 web-shell,需要维护者确认下游覆盖面。

方案:范围合理而非堆砌——每个机制都对应一个实测热点(派发批处理、50 ms 渲染节流 + 输入静默窗口、deferred 快照、COW 索引 + WeakMap 规范化工具内容、按"是否有内容"挂载计时器、有界的 Markdown 解析、仅尾部快路径),设计文档也明确列出了非目标(不做增量投影器、不用 Worker、不改 daemon 事件顺序)。代码审查阶段会重点看两处:投影快路径中未加解释的 '"insight_' 字符串守卫,以及 MessageList 中一组需要手动保持键检查与 memo 依赖列表同步的手写 ref 缓存。

风险:无升级风险信号——变更文件均未命中与 revert 相关的高风险路径。

进入代码审查 🔍 提示:由于 Stage 0 已提请维护者关注,无论审查结果如何,最终决定上限为转交维护者确认,不会自动批准。

Qwen Code · qwen3.8-max

Reviewed at 67a2de8aa2801a4a809c98e600c284b6eaba0707 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

🖼️ web-shell visual preview

Rendered against a mock daemon (no real backend): the PR base vs this PR head 08ad3ab. Only screenshots that changed are shown (flows below, if any, are head-only) — refreshes on every push.

Screenshots · before / after

⚠️ No preview: one or more scenarios failed to render on this head — see the workflow run. This is not "no visual change" — a scenario that times out or throws produces no image. Fix the failing scenario (or a genuine regression it caught) and the preview returns on the next push.

Full-resolution recordings (.webm) are attached to the workflow run.

Qwen Code · web-shell visuals

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Code review

My independent take on this problem would be: coalesce store notifications with a time-based, input-aware throttle; use useDeferredValue so keystrokes never queue behind transcript renders; extend the SDK's existing copy-on-write so unchanged history keeps stable identity; bound the repeated Markdown re-parse for large growing documents; and skip whole-list derivations when only the streaming tail grew. The PR does all of this — dispatch batching (16 ms), a 50 ms render throttle with an input-quiet window capped at 250 ms, deferred snapshots with session/reset staleness rejection, COW extended to blockIndexById, a WeakMap cache for normalized tool content, the 32 KB streaming plain-text bound, and tail-only fast paths in both the projection and MessageList. The design doc's non-goals (no incremental projector, no worker, no daemon ordering changes) show real restraint. I verified against base code: the sync-flush call sites that protect control/terminal ordering under the new 16 ms batch already exist, the mock-daemon e2e harness exports the spec uses all exist, and cross-env resolves from the root workspace.

No correctness blockers found. Findings worth the author's and maintainer's attention, none blocking:

  1. The '"insight_' guard in reuseUnchangedProjectedPrefix has no comment. This looks like the load-bearing correctness guard for the insight-payload splitter in transcriptToMessages (a completed insight_progress/insight_ready/insight_error JSON payload changes how one block projects into messages, so prefix reuse must be bypassed). That is exactly the kind of hidden constraint that deserves a one-line why comment — a future cleanup pass could plausibly delete it as a magic string. As written it is conservative (once the marker appears anywhere in the tail text, that stream stays on the full projection path — correct, just slower for insight streams).
  2. MessageList now carries five hand-rolled ref caches (mergedMessages, displayItems, finalAssistantTurnIds, visibleItems, visibleTurnIds) where each invalidation condition is expressed twice — once in the cache-key check and once in the useMemo dependency list. This is the known cost of tail fast paths (React's memo can't express "recompute everything except the tail"), and the abandoned-concurrent-render case is explicitly tested, but these lists must be kept in sync by hand and are the most likely future-bug surface in the diff. A maintainer should weigh this against the measured win before merge.
  3. Minor: visibleItems memo dependency moved from pendingApproval?.toolCallId to the whole pendingApproval object — harmless (identity only changes with the request itself) but slightly broader invalidation than before.

The test additions are genuinely pinning: identity preservation across streamed deltas, throttle window boundaries, input-quiet deferral with the 250 ms starvation cap, session-switch and same-session-reset staleness, abandoned concurrent renders, timer reuse, the plain-text → settled-Markdown transition, and the COW index sharing/copy contract at the SDK level.

sequenceDiagram
    participant P1 as Daemon SSE events
    participant P2 as DaemonSessionProvider
    participant P3 as Transcript store
    participant P4 as Render throttle hook
    participant P5 as useMessages projection
    participant P6 as MessageList fast path
    participant P7 as Composer input
    P1->>P2: chunk arrives
    P2->>P2: batch in a 16 ms window
    P2->>P3: one dispatch per window
    P3->>P4: notify
    P4->>P4: 50 ms throttle plus input quiet window
    P4->>P5: deferred snapshot, stale ones rejected
    P5->>P6: stable history prefix plus new tail
    P6->>P6: swap the tail row, keep virtualizer keys
    P7->>P4: beforeinput defers notify, capped at 250 ms
Loading
Files changed (24 of 24 shown)
File What changed
docs/design/web-shell-stream-render-performance.md New design doc — profiling evidence, six mechanisms, explicit non-goals
packages/sdk-typescript/scripts/build.js Comment-only note on the unchanged 197 KB browser bundle budget
packages/sdk-typescript/src/daemon/ui/transcript.ts Copy-on-write extended to the block index so text-only deltas keep it reference-stable, plus dev-mode freeze
packages/sdk-typescript/src/daemon/ui/types.ts blockIndexById typed as Readonly to match the COW contract
packages/sdk-typescript/test/unit/daemonUi.test.ts Index sharing on text deltas, copying on append, freeze behavior
packages/web-shell/client/adapters/messageTypes.ts Tool-call content typed readonly
packages/web-shell/client/adapters/transcriptToMessages.test.ts Stable normalized-content reference and renormalization on block replacement
packages/web-shell/client/adapters/transcriptToMessages.ts WeakMap cache of normalized tool content keyed by block identity
packages/web-shell/client/components/MessageList.dom.test.tsx DOM tests for the streamed-tail fast path, compact mode, earlier-row invalidation, abandoned concurrent renders
packages/web-shell/client/components/MessageList.tsx The five tail fast-path caches and identity-stable virtualizer keys/sizes
packages/web-shell/client/components/messages/AssistantMessage.test.tsx Timer survives streamed chunks and never starts for empty content
packages/web-shell/client/components/messages/AssistantMessage.tsx Elapsed-timer effect keyed on content presence instead of the growing string
packages/web-shell/client/components/messages/Markdown.module.css Styles for the streaming plain-text fallback
packages/web-shell/client/components/messages/Markdown.test.ts transformMarkdown still applied on the large-stream plain-text path
packages/web-shell/client/components/messages/Markdown.tsx Streaming content above 32 KB renders as escaped plain text until it settles
packages/web-shell/client/components/messages/MarkdownChartRenderer.test.tsx Large stream parses once as Markdown after settling
packages/web-shell/client/e2e/web-shell.stream-performance.spec.ts Opt-in deterministic perf scenario — 5000-turn history, 400 chunks, typing during stream, long-task and input-latency metrics
packages/web-shell/client/hooks/useAnimationFrameTranscriptBlocks.test.tsx Throttle window, input-quiet deferral, starvation cap, session switch, same-session reset
packages/web-shell/client/hooks/useAnimationFrameTranscriptBlocks.ts 50 ms render throttle, input-quiet window with 250 ms cap, deferred snapshot with staleness rejection
packages/web-shell/client/hooks/useMessages.test.ts Prefix identity preserved only for a streaming tail update, safe fallback on empty text
packages/web-shell/client/hooks/useMessages.ts Reuse the projected message prefix when only the streaming tail grew
packages/web-shell/package.json test:e2e:perf script gated on WEB_SHELL_PERF=1
packages/webui/src/daemon/session/DaemonSessionProvider.test.tsx Timer advances updated for the 16 ms batch
packages/webui/src/daemon/session/DaemonSessionProvider.tsx Transcript dispatch batched over one frame (0 → 16 ms) with existing sync flushes preserved

Testing evidence

This is an unattended CI run — PR code is never executed here. Evidence is the PR's own CI on the reviewed commit, fetched via the API at review time; the main unit suite is still running and the table updates in place when CI settles. No red checks at review time. Test (macos-latest) / Test (windows-latest) / integration jobs show skipped — that is the normal state for fork PRs in this repo (verified against the author's recently merged PR 9349, which shows the identical pattern), not a failure signal.

Final CI results for 67a2de8 (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Capture web-shell visuals (ubuntu-latest, Node 22.x) ✅ success
Classify PR ✅ success
Dependency CVE audit ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
Desktop Shell (windows-2022) ✅ success
Secret scan (TruffleHog) ✅ success
Test (ubuntu-latest, Node 22.x) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

What CI can and cannot settle here: the unit and DOM suites pin the mechanisms — identity preservation, throttle boundaries, staleness rejection, the plain-text → Markdown transition — and they run on the ubuntu job above. They do not pin the central claim, which is the performance win itself: the committed deterministic perf scenario is opt-in (WEB_SHELL_PERF=1) and does not run in normal CI, and the 17.8 s → 8.0 s / 92 → 1 long-task numbers are the author's own measurements on macOS only (author's claim, not independently re-run — and not verifiable from a diff read). Sandboxed verification would settle this: @qwen-code /verify — the author has write access, so it can be triggered directly, and an A/B load-bearing run against the base build (the committed deterministic scenario with WEB_SHELL_PERF=1 is the natural oracle) would show whether the stream-phase and long-task improvements actually hold and whether typed input survives the stream. /tmux is also available but the surface under review is browser UI, so /verify is the lane that fits the claim. Not verified: Windows/Linux behavior (author disclosed), and real-model streaming timing by design.

中文说明

代码审查

我对这个问题的独立方案是:用带时间窗口、感知输入的节流合并 store 通知;用 useDeferredValue 让按键永远不会排在 transcript 渲染之后;扩展 SDK 已有的 copy-on-write 让未变更历史保持稳定身份;对持续增长的大文档限制 Markdown 重复解析;以及当只有流式尾部增长时跳过全列表派生。PR 全部做到了——派发批处理(16 ms)、50 ms 渲染节流 + 上限 250 ms 的输入静默窗口、带会话/重置过期拒绝的 deferred 快照、COW 扩展到 blockIndexById、WeakMap 缓存规范化工具内容、32 KB 流式纯文本上限,以及投影层和 MessageList 的仅尾部快路径。设计文档明确的非目标(不做增量投影器、不用 Worker、不改 daemon 顺序)体现了克制。已对照基础代码验证:保护控制/终止事件顺序的同步 flush 调用点在 16 ms 批处理下已存在;e2e 场景用到的 mock-daemon 导出全部存在;cross-env 从根工作区解析。

未发现正确性阻塞问题。以下发现值得作者和维护者关注,均不阻塞:

  1. reuseUnchangedProjectedPrefix 中的 '"insight_' 守卫没有注释。 这看起来是 transcriptToMessages 中 insight 载荷拆分器的关键正确性守卫(完整的 insight JSON 载荷会改变一个 block 投影成消息的方式,因此必须绕过前缀复用)。这正是需要一行"为什么"注释的隐藏约束——未来某次清理可能会把它当魔法字符串删掉。当前实现偏保守(只要尾部文本中出现过该标记,该流就全程走完整投影路径——正确,只是 insight 流会慢一些)。
  2. MessageList 现在有五个手写 ref 缓存,每个失效条件都写了两遍——缓存键检查一遍、useMemo 依赖列表一遍。这是尾部快路径的已知代价(React memo 无法表达"除尾部外全部复用"),且并发渲染被放弃的场景有专门测试,但这些列表必须手工保持同步,是这份 diff 中最可能出未来 bug 的地方。合并前请维护者权衡实测收益。
  3. 小问题:visibleItems 的 memo 依赖从 pendingApproval?.toolCallId 改成了整个 pendingApproval 对象——无害(对象身份只随请求本身变化),但失效面略宽。

新增测试确实在"钉住"行为:流式增量间的身份保持、节流窗口边界、输入静默延迟与 250 ms 饥饿上限、会话切换和同会话重置的过期处理、被放弃的并发渲染、计时器复用、纯文本到 Markdown 的收敛过渡,以及 SDK 层 COW 索引的共享/复制契约。

(时序图见英文正文,流程为:SSE 事件 → 16 ms 批派发 → store → 50 ms 节流 + 输入静默窗口 → deferred 快照 → 稳定前缀投影 → 仅替换尾行;输入事件可延迟通知但上限 250 ms。)

测试证据

这是无人值守 CI 运行——此处绝不执行 PR 代码。证据为审查时通过 API 获取的该提交自身 CI 结果;主单测套件仍在运行,表格会在 CI 结束后原地更新。审查时无红色检查。Test (macos-latest) / Test (windows-latest) / 集成任务显示 skipped——这是本仓库 fork PR 的正常状态(已对照该作者近期合并的 PR 9349 验证,模式完全相同),不是失败信号。

CI 能钉住的是各机制(身份保持、节流边界、过期拒绝、纯文本→Markdown 过渡,见 ubuntu 任务);钉不住的是核心主张即性能收益本身:确定性 perf 场景是可选的(WEB_SHELL_PERF=1),普通 CI 不运行,17.8 s → 8.0 s / 92 → 1 的数字是作者在 macOS 上的自测(作者声明,非独立复现)。沙箱验证可以定论:@qwen-code /verify —— 作者有写权限,可直接触发;对基线构建的 A/B 负重验证(自带的 WEB_SHELL_PERF=1 确定性场景是天然 oracle)可以回答流式阶段和长任务的改善是否真实成立、流式期间键入是否不丢字。/tmux 也可用,但本次审查对象是浏览器 UI,/verify 才是与主张匹配的通道。未验证:Windows/Linux 行为(作者已声明),以及按设计不验证真实模型流的时序。

Qwen Code · qwen3.8-max

Reviewed at 67a2de8aa2801a4a809c98e600c284b6eaba0707 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Confidence: 3/5 — clean review with no blocking findings, but the Stage 0 escalation (cross-package core surface at ~723 production lines) caps this at a maintainer sign-off rather than an auto-approve.

Stepping back: this is a strong PR. The approach matches — and arguably exceeds — what I would have proposed independently, every mechanism traces to a measured hotspot, the design doc names its non-goals, and the test suite pins the actual invariants (identity, throttle bounds, staleness, fallbacks) rather than smoke. The author has a long, mostly-merged track record on exactly this surface, and the problem is real and user-facing, with direction corroborated by the reference product's changelog.

Why not approve from here:

  1. Policy cap. The change spans the sdk-typescript daemon/ui contract, webui's session provider, and web-shell — core infrastructure by this repo's definition — at ~723 production-logic lines. That combination routes the final call to a maintainer regardless of review cleanliness, and the SDK-level contract change (blockIndexById now Readonly and COW-tracked, consumed beyond this repo) is exactly the kind of thing a maintainer should confirm downstream coverage for.
  2. The central claim is still author-attested. The unit/DOM suites pin the mechanisms; the performance win itself (17.8 s → 8.0 s stream phase, 92 → 1 long tasks) rests on the author's macOS measurements, and the deterministic scenario that could reproduce it is opt-in. At review time the ubuntu unit suite and web-shell visuals were still in flight on this commit.

⏸️ Deferring to @chiga0 — needs a human call on two things: (a) sign-off on the daemon/ui contract change and the 16 ms dispatch batch for downstream SDK consumers, and (b) whether to require an A/B run first — @qwen-code /verify against the committed WEB_SHELL_PERF=1 scenario would turn the author's numbers into load-bearing evidence on this repo's runners. Non-blocking items for the author, foldable into a follow-up: comment the '"insight_' guard in useMessages.ts, and keep an eye on the five MessageList cache dependency lists.

No approval or rejection posted — the review stands on the two comments above.

中文说明

置信度:3/5 —— 审查干净、无阻塞发现,但 Stage 0 升级(跨核心包、约 723 行生产逻辑)将本 PR 的最终决定上限设为维护者签核,而非自动批准。

退一步看:这是一个高质量的 PR。方案与我独立设想的一致甚至更完整——每个机制都能追溯到实测热点,设计文档明确了非目标,测试钉住的是真实不变量(身份、节流边界、过期处理、兜底)而非表面行为。作者在这一领域有大量已合并的记录,问题真实且面向用户,方向也有参考产品 changelog 的佐证。

不在此批准的原因:

  1. 策略上限。 改动横跨 sdk-typescript daemon/ui 契约、webui 会话提供者和 web-shell——按本仓库定义属于核心基础设施——且生产逻辑约 723 行。这种组合无论审查多干净都要由维护者做最终决定;SDK 层契约变化(blockIndexById 变为 Readonly 并纳入 COW,消费方不止本仓库)正是需要维护者确认下游覆盖面的事情。
  2. 核心主张仍是作者自证。 单测/DOM 测试钉住了机制;性能收益本身(流式阶段 17.8 s → 8.0 s、长任务 92 → 1)依据作者在 macOS 上的测量,而能复现它的确定性场景是可选运行的。审查时该提交的 ubuntu 单测和 web-shell visuals 仍在进行。

⏸️ 转交 @chiga0 —— 需要人工决定两件事:(a) 对 daemon/ui 契约变化和 16 ms 派发批处理在下游 SDK 消费方上的影响签核;(b) 是否先要求一次 A/B 验证——对自带的 WEB_SHELL_PERF=1 场景运行 @qwen-code /verify,可以把作者的数字变成本仓库运行环境下的负重证据。给作者的非阻塞建议,可并入后续提交:为 useMessages.ts 中的 '"insight_' 守卫加注释;留意 MessageList 五个缓存的依赖列表同步。

未发布批准或拒绝——审查结论见上面两条评论。

Qwen Code · qwen3.8-max

Reviewed at 67a2de8aa2801a4a809c98e600c284b6eaba0707 · re-run with @qwen-code /triage

@ytahdn ytahdn left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

代码审查 / Code Review

结论 / Verdict: 💬 评论(无 blocking,倾向确认个别项后批准)/ Comment (no blockers; leaning approve after a couple of confirmations)


中文

总体印象:这是一个设计扎实、测试覆盖充分的性能优化。设计文档清晰说明了问题、方案与取舍,确定性负载下的前后对比数据可信(流式阶段 17.8s→8.0s、长任务 92→1)。优化手段(批量投递、useDeferredValue 延迟渲染、COW 身份稳定、Markdown 解析预算)都针对实测热点,而非凭直觉。

做得好的地方 🎉

  • useAnimationFrameTranscriptBlocksuseDeferredValue + sessionId/blockIndexById 双身份兜底,既保输入流畅,又从根上杜绝“重置后显示已丢弃 transcript / 切会话串数据”,配套测试到位。
  • Markdown 大流降级为纯文本走 React 文本插值,自动转义、无 XSS;transformMarkdown 自定义钩子在纯文本模式下仍被调用,可定制性没丢。
  • 边界覆盖用心:abandoned concurrent render 不污染缓存、undefined/空 content 安全降级、ThinkingMessage 计时器不再每 chunk 重建 interval。
  • 已核实transcript.ts 中 blocks 与 index 的所有权拆分是完整正确的——唯一直接写 index 的 appendBlock 已调用 takeBlockIndexOwnership,其余路径均整体 rebuild 并注册所有权,getWritableBlockById 的原地替换不改变 index。我最初担心的“遗漏写点导致 COW index 跨快照污染”不成立。

行内评论(见 line comments)

  1. 🟡 useMessages.ts"insight_" 魔法字符串需解释/具名化。
  2. 🟡 MessageList.tsx — render 期写 ref 的反模式,建议固化不变量/收敛为 hook。
  3. 🟢 package.json — 确认 cross-env 已在 devDependencies。

建议 💡

  • Markdown:流结束瞬间会对超大文档做一次完整解析,可能产生单次卡顿;设计文档已列为非目标,未来可考虑分片解析。
  • e2e perf spec:目前只做正确性断言 + 记录指标,不设性能阈值,无法在 CI 中作为回归门禁;可考虑加软阈值或趋势对比。
  • transcriptToMessagesObject.freeze(content) 只冻结外层数组;WeakMap 按 block 引用缓存依赖“store 永不原地改 block”这一约定,建议在注释里写明。

English

Overall: A well-designed, thoroughly tested performance optimization. The design doc clearly states the problem, approach, and trade-offs, and the deterministic before/after numbers are credible (stream phase 17.8s→8.0s, long tasks 92→1). The techniques (batched delivery, useDeferredValue deferral, COW identity stability, Markdown parse budget) target measured hotspots rather than guesswork.

What's done well 🎉

  • useAnimationFrameTranscriptBlocks uses useDeferredValue with a sessionId/blockIndexById dual-identity fallback: keeps input fluid while structurally preventing "showing discarded transcript after reset" and cross-session bleed; tests cover both.
  • The Markdown large-stream plain-text fallback uses React text interpolation (auto-escaped, no XSS), and still invokes the transformMarkdown customization hook in plain-text mode.
  • Careful edge coverage: abandoned concurrent renders don't pollute caches, undefined/empty content degrades safely, and the ThinkingMessage timer no longer recreates its interval per chunk.
  • Verified: the blocks/index ownership split in transcript.ts is complete and correct — appendBlock (the only direct index writer) calls takeBlockIndexOwnership, all other paths rebuild the index wholesale and register ownership, and getWritableBlockById's in-place replacement doesn't change the index. My initial worry about a missed write site poisoning a shared COW index does not hold.

Inline comments (see line comments)

  1. 🟡 useMessages.ts — the "insight_" magic string needs an explanation / a named constant.
  2. 🟡 MessageList.tsx — ref writes during render are an anti-pattern; please pin down the invariants / collapse into a hook.
  3. 🟢 package.json — confirm cross-env is already a devDependency.

Suggestions 💡

  • Markdown: the one-time full parse of a very large document at stream end may jank; acknowledged as a non-goal, but consider chunked parsing later.
  • e2e perf spec: it only asserts correctness and records metrics, with no perf threshold, so it can't gate regressions in CI; consider a soft threshold or trend comparison.
  • transcriptToMessages: Object.freeze(content) freezes only the outer array; the block-keyed WeakMap cache relies on the "store never mutates a block in place" contract — worth spelling out in a comment.

Comment thread packages/web-shell/client/hooks/useMessages.ts Outdated
Comment thread packages/web-shell/client/components/MessageList.tsx
Comment thread packages/web-shell/package.json
@ytahdn
ytahdn requested a review from wenshao August 18, 2026 12:09

@chiga0 chiga0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: perf(web-shell): keep streaming output responsive

This is a well-scoped performance PR applying several complementary techniques — 16 ms transcript-event batching, 50 ms render throttle with input-quiet window, useDeferredValue with session/blockIndex identity guard, stable projection-prefix identity, WeakMap tool-content caching, and a streaming Markdown size cap with plain-text fallback. The architecture is sound and the 258-line e2e perf spec (@perf) gives solid regression coverage.

Two minor findings (both flagged in the author's own self-review, confirmed below) need attention before merge.


Cross-Validation

# Finding Author My Assessment
C1 useMessages.ts:107'"insight_' magic string; no comment explaining why it disqualifies prefix reuse ytahdn ✅ Confirmed — inline comment posted. Named constant + explanation required; correctness-guard vs. heuristic distinction is unclear.
C2 MessageList.tsx:3233 — multiple *Cache.current / reusedVisibleStreamingTailRef.current written during useMemo render phase ytahdn ✅ Confirmed — inline comment posted. Safe via previousMessagesRef / useLayoutEffect guard (tests validate), but the invariant must be documented in-code for future maintainers.
C3 web-shell/package.json:29cross-env used in test:e2e:perf without explicit dep declaration ytahdn ❓ (verify) Non-issuecross-env@^7.0.3 is declared in the monorepo root package.json#devDependencies and is hoisted. No per-package redeclaration needed.

Additional Audit Coverage

blockIndexById COW ownership — Traced all mutation sites: appendBlock (calls takeBlockIndexOwnership ✓), discardToolBlock (rebuilds index + registers ownership ✓), trimTranscriptState (rebuilds + registers ✓), truncateTranscriptBeforeBlock (calls rebuildTranscriptIndexes + registers ✓). getWritableBlockById mutates only block content, leaving the index reference untouched (correct: no index entry changes on content-only edits). Object.freeze(result.blockIndexById) in FREEZE_TRANSCRIPT_COLLECTIONS seals the object for downstream consumers. Ownership accounting correct throughout.

useDeferredValue session/blockIndex guard — Verified the two-condition guard (deferred.sessionId === sessionId && deferred.blockIndexById === live.blockIndexById) covers all transitions:

  • Session switch → sessionId mismatch → fall through to live.blocks immediately ✓
  • Same-session store reset (e.g., truncation) → new blockIndexById object → mismatch → live ✓
  • Streaming text growth on existing block → takeBlocksOwnership only; blockIndexById reference unchanged → deferred path active (input stays responsive) ✓
  • New block appended → takeBlockIndexOwnership called → new object → live ✓ (structural change, not just streaming text)

Markdown plain-text XSS safety<pre>{renderedContent}</pre> renders renderedContent as a React text node (not dangerouslySetInnerHTML). React escapes all HTML entities. The transformMarkdown result is also text-interpolated, not injected as raw HTML. No XSS risk.

ThinkingMessage timer stabilizationuseEffect dependency changed from content (string) to hasContent (boolean). Timer interval is now created/destroyed only when thinking starts or stops, not on every streaming token. Correct optimization; no timer leak path.

reuseUnchangedProjectedPrefix computational costtranscriptBlocksToLocalizedMessages(blocks, t) is always called before the fast path can skip downstream work. The O(transcript) projection is paid on every render; what the prefix-reuse buys is stable React object identity for unchanged history messages (avoiding downstream re-renders) and cache hits in MessageList.tsx for mergeCompactToolGroups / attachTurnOutputs / applyTurnCollapse. Consistent with the PR's own profiling note that projection contributes only ~2.5% of main-thread time — the list-processing pipeline is the real target.


Verdict

COMMENT — two minor findings (C1 and C2), both non-blocking on correctness but important for long-term maintainability. Core perf strategy is architecturally sound; no correctness, security, or regression issues found.

Comment thread packages/web-shell/client/hooks/useMessages.ts Outdated
Comment thread packages/web-shell/client/components/MessageList.tsx

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no blockers. Suggestions are inline.

中文说明

已审查——无阻断问题。 建议见行内评论。

— qwen3.8-max via Qwen Code /review (v0.21.13)

Comment thread packages/web-shell/client/components/MessageList.tsx Outdated
Comment thread packages/web-shell/client/e2e/web-shell.stream-performance.spec.ts Outdated
Comment thread packages/web-shell/client/hooks/useMessages.ts Outdated
Comment thread packages/webui/src/daemon/session/DaemonSessionProvider.tsx
Comment thread packages/web-shell/client/components/MessageList.tsx

@chiga0 chiga0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Overview (AI Generated)

PR: #9405 perf(web-shell): keep streaming output responsive
Type: Performance Optimization
Change size: +1680/-109 across 24 files
HEAD: 08ad3ab8d


Findings Summary

  • Critical/Major: 0 (all prior findings resolved at HEAD)
  • Minor/Nit: 2 (new, non-blocking)

Key Observations

A well-designed, thoroughly benchmarked performance PR. The stack of complementary techniques — 16 ms event batching, 50 ms render throttle with input-quiet window, useDeferredValue + session/blockIndexById identity guard, stable projection-prefix identity, WeakMap tool-content caching, and a 32 KB Markdown parse cap — collectively reduce the stream phase from 17.8 s to ~8 s and long-task count from 92 to 1 under a deterministic 5000-turn / 400-chunk workload. All critical findings from earlier rounds (timestamp guards disabling fast paths in production, E2E promptId mismatch, older-history anchor race, INSIGHT_CONTENT_MARKER undocumented, render-phase ref-write invariant undocumented, 16 ms batch untested) are confirmed fixed at HEAD.


Cross-Validation

# Finding Prior Review My Assessment
C1 useMessages.tsserverTimestamp guard in reuseUnchangedProjectedPrefix disabled prefix reuse on every live delta qwen-code-ci-bot R1-3 ✅ Confirmed fixed — guard removed at HEAD; regression test explicitly varies serverTimestamp: 1_001→1_002 and verifies fast path still fires
C2 MessageList.tsxtimestamp comparison in isStreamingTailContentOnlyUpdate disabled merged/display/visible fast paths on every live frame qwen-code-ci-bot R1-1 ✅ Confirmed fixed — comparison removed; DOM regression test with timestamp 1001→1002 confirms fast path engages
C3 web-shell.stream-performance.spec.ts — terminal event promptId: 'performance-prompt' mismatched mock daemon's assigned id; turn never settled qwen-code-ci-bot R1-2 ✅ Confirmed fixed — promptId corrected to 'prompt-e2e'; streaming-plain-text-disappears assertion added to cover turn-settle render pass
C4 DaemonSessionProvider.tsx:832 — 16 ms batch window untested; setTimeout(0) revert would be invisible qwen-code-ci-bot R1-4 ✅ Confirmed fixed — fake-timer test added: two chunks 5 ms apart, no dispatch before 16 ms, single size-2 batch after window closes
C5 MessageList.tsx — older-history anchor cleared after one frame before parent could commit prepended messages (race → wrong scroll position) wenshao CR ✅ Confirmed fixed — waitForPrepend loop waits for message-count change, 30-frame upper bound, new-load cancels pending frame; regression test with 2-frame prepend delay added
C6 useMessages.tsINSIGHT_CONTENT_MARKER literal undocumented ytahdn self-review / chiga0 ✅ Confirmed fixed — named constant extracted; comment explains insight JSON projects one growing text block into multiple messages, making prefix reuse unsafe
C7 MessageList.tsx:3239 — render-phase cache writes (*Cache.current, reusedVisibleStreamingTailRef) lack invariant documentation ytahdn self-review / chiga0 ✅ Confirmed fixed — inline comment explains these caches are keyed to previousMessagesRef (post-commit identity), so abandoned renders cannot advance the guard and their writes are harmlessly discarded by the next committed render
C8 web-shell/package.jsoncross-env used in new perf script without local dep declaration ytahdn self-review ✅ Non-issue — cross-env@^7.0.3 declared in repo-root package.json devDependencies, hoisted across workspace

New Findings (non-blocking)

Nit-1: mergedMessages fast path has an assistant-only role guard while isStreamingTailContentOnlyUpdate returns true for both assistant and thinking tails. For thinking streams, streamingTailContentOnly === true but mergedMessages and displayItems fast paths both gate on tail?.role === 'assistant' and fall through to full recomputation. The optimization is skipped for the thinking-stream case — correct behavior, but a minor performance gap that could be closed in a follow-up.

Nit-2: normalizedToolContentCache (module-level WeakMap) freezes only the outer array (Object.freeze(content)), not its DaemonMessageToolCallContent elements. The cache correctly relies on the invariant that the transcript store never mutates a block in place (COW discipline). The comment added in discardToolBlock/trimTranscriptState about ownership tracking is good; a parallel one-liner in transcriptToMessages.ts noting the shallow-freeze and the COW assumption would close the doc gap ytahdn flagged.


Additional Audit Coverage

Areas independently verified beyond existing findings:

  • blockIndexById COW ownership completeness — traced all 4 mutation paths: appendBlock (calls takeBlockIndexOwnership ✓), discardToolBlock (rebuilds + registers ✓), trimTranscriptState (rebuilds + registers ✓), truncateTranscriptBeforeBlock (calls rebuildTranscriptIndexes then registers ✓). getWritableBlockById mutates only block fields, not the index — no ownership call needed. ✓
  • hasPendingInput() API availabilitynavigator.scheduling.isInputPending() is Chromium-only; the optional-chaining fallback correctly treats absence as false (no pending input), which keeps the throttle conservative on Firefox/Safari. ✓
  • renderStreamingPlainText with transformMarkdownrenderedContent is the source-transformed text (not raw content). If a custom transformMarkdown modifies the string, the <pre> displays the transformed version. This is intentional — the transformMarkdown hook applies before the Markdown renderer and the plain-text path preserves that contract. ✓
  • normalizeToolContent WeakMap GC safety — module-level WeakMap keyed by block objects. Blocks are garbage-collected with the transcript, so no permanent retention. Keys never collide because COW gives each mutated block a new identity. ✓
  • pendingSinceTs max-deferral reset — set when the first rAF is scheduled, cleared when dispatch fires; re-schedules inside dispatchWhenDue do not reset it, so the 250 ms cap is measured from first deferral, not from each rescheduled frame. ✓
  • clientReceivedAt vs serverTimestamp stabilityclientReceivedAt is set once when a block first arrives; serverTimestamp was updated on every SSE delta (now removed). The surviving clientReceivedAt guard is correct and stable. ✓

Final Verdict

Approve. All critical and suggestion findings are confirmed resolved at HEAD. The performance improvements are substantial (2× stream-phase speedup, 92→1 long tasks), the test coverage is thorough, and the two remaining Nits are cosmetic follow-up candidates. Ready to merge.


This review was generated by QoderWork AI

@ytahdn
ytahdn added this pull request to the merge queue Aug 19, 2026
Merged via the queue into QwenLM:main with commit 3192323 Aug 19, 2026
232 of 233 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants