fix(server): speed up long thread message sync

[Yash-Singh1]

Problem

Long forked threads can spend more than two seconds syncing messages before the client shows history. Profiling a long thread against a 57 GB database found repeated visibility work, unnecessary ancestor hydration, extra payload lookups, and repeated serialization in the bounded snapshot path.

Change

• Build the existing visibility lookup once and calculate local visibility once per projection.
• Load only timeline rows, runs, and attempts from fork ancestors during bounded reads.
• Sort retained node IDs before fetching payloads through the existing primary key, then restore their order without sorting large JSON bodies in SQLite.
• Carry SQLite row IDs through timeline selection to avoid a second string-ID index lookup for every payload.
• Skip the final full-projection budget serialization when the already-measured control state exceeds the cap.

Uses existing indexes. No migrations, schema changes, new dependencies, or response-format changes.

Scope and approval

Requested through the T3 Code development thread to investigate message-sync latency and open this draft. These changes address the same server snapshot-read bottleneck and preserve history paging, visibility, and actionable state.

Verification

Combined benchmark

Compared the starting implementation with all changes together on the same frozen database, using four warmup pairs and 15 measured alternating pairs per thread. Timings include SQL reads, decoding, wire projection, bounded selection, schema encoding, and JSON serialization. They exclude process startup, network transfer, and client rendering. Every encoded response body matched byte for byte.

Thread	Starting median	Current median	Reduction
Long forked thread	395.6 ms	303.5 ms	23%
Medium thread	176.6 ms	140.4 ms	21%
Shorter thread	40.7 ms	41.1 ms	Essentially unchanged

The long response remains 12.46 MB. Its measured p90 preparation time fell from 544 to 400 ms.

Live database and component measurements

Seven alternating fresh processes per query version on the real database, opened read-only, measured median snapshot reads of 2,123 to 927 ms and response preparation of 2,428 to 1,601 ms, a 34% response-time reduction. This comparison preceded the final timeline row-ID change and used the new budget calculation on both sides. OS caches were not cleared, live data could change between processes, and timings varied widely. Response equivalence was checked separately within pinned read transactions.

Independent component comparisons measured payload-budget calculation at 99.5 to 25.3 ms across 15 alternating pairs on frozen data, and total timeline SQL at 49.5 to 41.3 ms across nine alternating fresh connections with warm OS caches. These percentages describe different measurements and should not be added.

The production client still reproduced a 2.09-second click-to-sync-completion wait with optimized code against an isolated copy of the long thread. Optimized fresh server responses in the earlier live test ranged from 0.78 to 4.13 seconds. This does not establish a matching UI speedup or eliminate every slow first read.

Correctness checks

• 112 focused tests passed across ProjectionStore, boundedSnapshotTransport, ThreadStream, threadHistoryPaging, WireProjection, and orchestrationV2Timeline.
• Query-plan checks confirm indexed user-message boundary selection, direct row-ID timeline payload reads, direct node primary-key reads, and no final payload sort.
• Coverage includes complete user turns, equal-ordinal ID ordering with reversed insertion order, null/equal node timestamps, duplicate node references, rollback and interrupt visibility, nested forks, cursor paging, and oversized actionable state.
• 30 full/window/history-anchor snapshot comparisons were byte-identical. The budget shortcut also produced 68 byte-identical bounded responses across paging configurations and payload caps.
• Server typecheck, targeted lint and formatting, and git diff checks passed.

Focused test command:

vp test run apps/server/src/orchestration-v2/ProjectionStore.test.ts apps/server/src/orchestration-v2/boundedSnapshotTransport.test.ts apps/server/src/orchestration-v2/ThreadStream.test.ts apps/server/src/orchestration-v2/threadHistoryPaging.test.ts apps/server/src/orchestration-v2/WireProjection.test.ts packages/shared/src/orchestrationV2Timeline.test.ts

Implemented with GPT-6.1-sol through the Codex harness in T3 Code.

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	5.0 KiB	5.0 KiB	0 B (0.0%)	6.8 KiB	✅
Codex	Thread snapshot wire	3.8 KiB	3.8 KiB	0 B (0.0%)	4.9 KiB	✅
Codex	Live turn WebSocket wire	1.2 KiB	1.2 KiB	0 B (0.0%)	2.0 KiB	✅
Codex	Live turn WebSocket decoded	20.9 KiB	20.9 KiB	0 B (0.0%)	29.3 KiB	✅
Codex	Live turn messages	2	2	0 (0.0%)	8	✅
Claude	Total thread wire	5.0 KiB	5.0 KiB	0 B (0.0%)	6.8 KiB	✅
Claude	Thread snapshot wire	3.8 KiB	3.8 KiB	0 B (0.0%)	4.9 KiB	✅
Claude	Live turn WebSocket wire	1.2 KiB	1.2 KiB	0 B (0.0%)	2.0 KiB	✅
Claude	Live turn WebSocket decoded	21.2 KiB	21.2 KiB	0 B (0.0%)	29.3 KiB	✅
Claude	Live turn messages	2	2	0 (0.0%)	8	✅

Baseline: b707eeb · PR result: bc7cead · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 108.5 KiB
• Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Path: .coderabbit.config.ts
• Review profile: CHILL
• Plan: Team
• Run ID: fd791855-94df-42b9-9b43-ef5cdb9c2920

📥 Commits

Reviewing files that changed from the base of the PR and between b707eeb and bc7cead.

📒 Files selected for processing (5)

• apps/server/src/orchestration-v2/ProjectionStore.test.ts
• apps/server/src/orchestration-v2/ProjectionStore.ts
• apps/server/src/orchestration-v2/WireProjection.test.ts
• apps/server/src/orchestration-v2/threadHistoryPaging.test.ts
• apps/server/src/orchestration-v2/threadHistoryPaging.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

Projection reads now apply shared turn-item visibility and use revised bounded SQL queries for timeline items and nodes. Thread history paging also marks the payload budget as exceeded when the control projection alone exceeds the configured byte limit.

Changes

Projection reads and bounded paging

Layer / File(s)	Summary
Projection visibility and fork inputs apps/server/src/orchestration-v2/ProjectionStore.ts, apps/server/src/orchestration-v2/WireProjection.test.ts	Local turn items use the shared visibility predicate. Canonical reads no longer append local visible items, and windowed fork reads request only runs, attempts, and turn items from ancestors.
Bounded window selection and node reads apps/server/src/orchestration-v2/ProjectionStore.ts, apps/server/src/orchestration-v2/ProjectionStore.test.ts	Window queries carry turn-item rowids into payload retrieval. Node reads fetch payloads by primary key in retained-node order. Tests cover ordering, payloads, and SQL plans.
Thread history payload budget apps/server/src/orchestration-v2/threadHistoryPaging.ts, apps/server/src/orchestration-v2/threadHistoryPaging.test.ts	The exceeded-budget flag now includes the control projection size. The test checks that bounded paging retains visible timeline items.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to bc7ce

No actionable issue is established that would prevent merging after normal checks.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: improving server performance for long thread message synchronization.
Description check		The description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the bottleneck, implementation, scope, benchmark results, correctness checks, limitati…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Approved at bc7cead

Macroscope's review found this PR approvable — This is a contained server-side performance fix that optimizes indexed history and node reads while preserving projection contents, ordering, and wire output. The accompanying tests cover the affected paging, fork, visibility, query-plan, and budget paths without changing product defaults or introducing new capabilities.

^{You can add or adjust custom eligibility rules. Learn more.}