Skip to content

fix(server): held queued wakes no longer keep delegated tasks running - #17028

Merged
juliusmarminge merged 1 commit into
mainfrom
t3code/fix/held-queued-wakes-lifecycle
Oct 8, 2026
Merged

juliusmarminge merged 1 commit into
mainfrom
t3code/fix/held-queued-wakes-lifecycle

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Problem

A delegated child thread's queued runs (for example pull request watch wakes) get held for the user when the child is stopped, the server restarts, or the provider fails. Those held runs were still counted as active work. As a result:

  • delegatedTaskProgress kept the task in working, so the parent's subagent row stayed running with no result.
  • Startup subagent-results recovery (TS predicate and SQL) skipped the child, so a restart did not repair it either.
  • task_status reported hasPendingChildRuns: true for turns that only the user can resume.

There was a second ordering bug in handleTerminalRun. It finalized the child's task before startNextQueuedRun held the queue after a provider failure. Finalization still saw a deliverable queued wake, returned working, and nothing finalized the task again until the next restart.

Before: a stopped or failed child with a held wake leaves the parent task stuck "running" indefinitely.
After: the task settles with the cancelled or failed run as its result, and the held wake stays held for the user to resume or discard.

Fix

  • A queued run with queueHeld === true no longer counts as owed work in delegatedTaskProgress, needsRecovery, the recovery SQL (queueHeld IS 1, so missing or NULL is treated as unheld), and hasPendingChildRuns. The task_status tool description now says so.
  • handleTerminalRun makes the queue decision first and finalizes afterwards. A failure in the queue step is logged and no longer skips finalization. Locks are still taken sequentially, never nested, so lock order is unchanged.

The scope was authorized by a maintainer and stays inside the server's subagent lifecycle. There are no contract, client, or provider-adapter changes.

Verification

The proof is deterministic tests. Screenshots don't apply because no UI changed.

  • With the new tests in place and the source change reverted, 5 tests fail:
    • settles a cancelled child whose held wakes wait behind it
    • settles a child whose provider failure holds its queued wakes (times out: the task stays running)
    • recovers terminal subagent results until their cross-thread transfer exists
    • waits for nested work and retains the report across monitor acknowledgements
    • the MCP toolkit hasPendingChildRuns case
  • With the fix, 11 related suites pass, 185/185 tests: delegated delivery, subagent projection, projection recovery and store, the MCP toolkit, restart continuation, thread stop, background work stop, runtimeLayer, provider runtime recovery, and run completion reads.
  • tsc --noEmit in apps/server is clean. Lint on the touched files is clean apart from one existing unused-variable warning that this change doesn't touch.
  • An independent review approved the fix and contributed the ordering correction.
  • Both failure modes were reproduced from a read-only snapshot of real data. Those records are private and not attached.

The fix only reaches existing stuck tasks after the host runs this build and startup recovery runs.

Done by Claude Opus 5.5 in T3 Code (Claude Code harness).

🤖 Generated with Claude Code


Devin Review

A child thread's queued runs that Stop, a restart, or a provider failure held
for the user were still counted as active work. Delegated task finalization,
startup subagent-result recovery, and task_status hasPendingChildRuns all
treated the child as busy, so the parent's task row stayed "running" with no
result.

Held queued runs now wait for the user instead of counting as owed work, in
delegatedTaskProgress, the recovery predicate (TS and SQL), and
hasPendingChildRuns. handleTerminalRun also makes the queue decision before
finalizing the child's task, so a provider failure holds the queue first and
the failed run becomes the result; a queue failure is logged and no longer
skips finalization.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Oct 8, 2026
@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 8, 2026
@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Team
  • Run ID: bb5a7be5-e94a-4320-8e48-166e887d7875
📥 Commits

Reviewing files that changed from the base of the PR and between d720210 and ac10d11.

📒 Files selected for processing (9)
  • apps/server/src/mcp/OrchestratorMcpService.ts
  • apps/server/src/mcp/OrchestratorMcpToolkit.integration.test.ts
  • apps/server/src/mcp/toolkits/orchestrator/tools.ts
  • apps/server/src/orchestration-v2/DelegatedCompletionDelivery.test.ts
  • apps/server/src/orchestration-v2/Orchestrator.ts
  • apps/server/src/orchestration-v2/ProjectionRecovery.test.ts
  • apps/server/src/orchestration-v2/ProjectionStore.ts
  • apps/server/src/orchestration-v2/SubagentProjection.test.ts
  • apps/server/src/orchestration-v2/SubagentProjection.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.


📝 Walkthrough

Walkthrough

The change excludes queued child runs marked queueHeld from pending-work and recovery calculations. For run.updated events, delegated completion delivery now runs before queued-run startup, whose failures are logged and swallowed.

Changes

Held queued child run handling

Layer / File(s) Summary
Pending child work classification
apps/server/src/orchestration-v2/SubagentProjection.ts, apps/server/src/orchestration-v2/SubagentProjection.test.ts, apps/server/src/mcp/OrchestratorMcpService.ts, apps/server/src/mcp/OrchestratorMcpToolkit.integration.test.ts, apps/server/src/mcp/toolkits/orchestrator/tools.ts
Progress and MCP pending-run checks exclude queued runs marked queueHeld. Tests cover held and unheld queued runs. The task status description documents that held turns wait for the user and do not count as pending child runs.
Subagent recovery selection
apps/server/src/orchestration-v2/ProjectionStore.ts, apps/server/src/orchestration-v2/ProjectionRecovery.test.ts
Recovery checks use the latest unheld run. Tests cover recovery behavior for held, deliverable-queued, and other child states.
Terminal run completion delivery
apps/server/src/orchestration-v2/Orchestrator.ts, apps/server/src/orchestration-v2/DelegatedCompletionDelivery.test.ts
run.updated handling finalizes delegated completion delivery before attempting queued-run startup. Startup failures are logged and swallowed. Tests cover restart recovery when held queued runs remain.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related PRs

  • pingdotgg/t3code#15170: Introduced delegated-task reconciliation and terminal handling in the same paths affected by this change.

Suggested reviewers: maria-rcks, t3dotgg

Merge Risk: ⚪ Minimal · up to ac10d

This change stops held queued runs from keeping delegated tasks in the running state. No merge-blocking risk was identified.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the primary fix: held queued wakes no longer keep delegated tasks running.
Description check ✅ Passed The description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the affected behavior, implementation, authorization, tests, type-checking, lint statu…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 20.9 KiB 20.9 KiB 0 B (0.0%) 29.3 KiB ✅
Codex Live turn messages 2 2 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.2 KiB 21.2 KiB 0 B (0.0%) 29.3 KiB ✅
Claude Live turn messages 2 2 0 (0.0%) 8 ✅

Baseline: d720210 · PR result: ac10d11 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@macroscopeapp

macroscopeapp Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at ac10d11

Macroscope's review found this PR approvable — This is a focused server-side bug fix that prevents user-held queued wakes from falsely keeping delegated tasks active and ensures provider-failure finalization occurs in the correct order. The runtime edits are narrow, consistent across status and recovery paths, and accompanied by targeted regression coverage.

You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarminge merged commit 0bfd9dd into main Oct 8, 2026
33 of 34 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/fix/held-queued-wakes-lifecycle branch October 8, 2026 04:49
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Oct 8, 2026
## What's Changed
* chore(review): remove the custom Approvability check by @esthor in pingdotgg/t3code#17018
* perf(mobile): show the cached thread list sooner and stop the freeze after it by @juliusmarminge in pingdotgg/t3code#16713
* fix(server): replaying a command no longer freezes the server by @SunkenInTime in pingdotgg/t3code#17041
* fix(server): opening a long thread no longer blocks the server for a second by @SunkenInTime in pingdotgg/t3code#17029
* fix(server,web,mobile): thread links reference the thread id, not a baked title by @juliusmarminge in pingdotgg/t3code#17017
* fix(web): messages with quotes or links no longer collapse when short by @flamboh in pingdotgg/t3code#16624
* refactor(web): PR loading skeleton shares the detail panel's layout by @flamboh in pingdotgg/t3code#15583
* fix(web): PR file headers keep the file name in narrow panels by @flamboh in pingdotgg/t3code#15562
* fix(web): PR timeline no longer shifts when its scrollbar appears by @flamboh in pingdotgg/t3code#15588
* fix(server): held queued wakes no longer keep delegated tasks running by @juliusmarminge in pingdotgg/t3code#17028
* fix(server): a message sent during a rollback no longer undoes it by @t3dotgg in pingdotgg/t3code#17079
* fix(web): PR file stats ignore the hide-whitespace toggle by @flamboh in pingdotgg/t3code#16162
* perf(mobile): omit duplicated turn items from bounded thread snapshots by @juliusmarminge in pingdotgg/t3code#15385
* perf(mobile): pause elapsed-time timers on hidden thread screens by @juliusmarminge in pingdotgg/t3code#15397
* fix(mobile): pause hidden home thread list updates by @juliusmarminge in pingdotgg/t3code#15705
* fix(mobile): restore file viewer insets and glass header by @juliusmarminge in pingdotgg/t3code#17073
* fix(web): PR code toolbar no longer overlaps in narrow panels by @flamboh in pingdotgg/t3code#15561
* fix(web): PR commit menu no longer stretches across the window by @flamboh in pingdotgg/t3code#15560
* fix(web): command palette scrollbar no longer clipped at the top by @flamboh in pingdotgg/t3code#17035
* fix(web): Usage breadcrumb stays centered on small viewports by @flamboh in pingdotgg/t3code#15552
* fix(mobile): stop refreshing Git status on streamed thread updates by @juliusmarminge in pingdotgg/t3code#15893
* fix(mobile): skip move indexes for empty and single-thread sections by @juliusmarminge in pingdotgg/t3code#16115
* perf(mobile): reuse encoded rows in shell cache saves by @juliusmarminge in pingdotgg/t3code#16129
* perf(mobile): skip showcase subscriptions in normal builds by @juliusmarminge in pingdotgg/t3code#16131
* perf(mobile): remove unused Home project sorting by @juliusmarminge in pingdotgg/t3code#16177
* perf(mobile): reduce move-menu index allocations by @juliusmarminge in pingdotgg/t3code#16256
* perf(mobile): skip impossible thread-key lookups by @juliusmarminge in pingdotgg/t3code#16263
* fix(mobile): collect UI runtime garbage on iOS memory warnings by @juliusmarminge in pingdotgg/t3code#16296
* fix(mobile): stop Git sheet refresh loop by @juliusmarminge in pingdotgg/t3code#16305
* fix(mobile): refresh Git status after reconnect by @juliusmarminge in pingdotgg/t3code#16329
* fix(mobile): update the iOS Git header menu when status changes by @juliusmarminge in pingdotgg/t3code#16330
* perf(mobile): reuse the settled sort when settled rows are unchanged by @juliusmarminge in pingdotgg/t3code#16369
* perf(mobile): highlight source files in small batches that keep grammar state by @juliusmarminge in pingdotgg/t3code#16729
* feat(web): quote chips show what you said about the quote by @flamboh in pingdotgg/t3code#15703
* feat(web): projectless threads show their machine in the sidebar by @flamboh in pingdotgg/t3code#17022
* fix(lineage): keep agent effort and speed after completion by @Bil0000 in pingdotgg/t3code#16925
* fix(mobile): align diff scrolling with glass headers by @juliusmarminge in pingdotgg/t3code#17085
* fix(web): workspace page headers can no longer grow past the top-bar height by @maria-rcks in pingdotgg/t3code#17086
* feat(mobile): redesign the Add environment sheet by @juliusmarminge in pingdotgg/t3code#17092


**Full Changelog**: pingdotgg/t3code@v0.0.46-nightly.20261008.2801...v0.0.46-nightly.20261008.2813

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.46-nightly.20261008.2813
github-actions Bot added a commit to davidvanderklay/t3code-flake that referenced this pull request Oct 8, 2026
## What's Changed
* chore(review): remove the custom Approvability check by @esthor in pingdotgg/t3code#17018
* perf(mobile): show the cached thread list sooner and stop the freeze after it by @juliusmarminge in pingdotgg/t3code#16713
* fix(server): replaying a command no longer freezes the server by @SunkenInTime in pingdotgg/t3code#17041
* fix(server): opening a long thread no longer blocks the server for a second by @SunkenInTime in pingdotgg/t3code#17029
* fix(server,web,mobile): thread links reference the thread id, not a baked title by @juliusmarminge in pingdotgg/t3code#17017
* fix(web): messages with quotes or links no longer collapse when short by @flamboh in pingdotgg/t3code#16624
* refactor(web): PR loading skeleton shares the detail panel's layout by @flamboh in pingdotgg/t3code#15583
* fix(web): PR file headers keep the file name in narrow panels by @flamboh in pingdotgg/t3code#15562
* fix(web): PR timeline no longer shifts when its scrollbar appears by @flamboh in pingdotgg/t3code#15588
* fix(server): held queued wakes no longer keep delegated tasks running by @juliusmarminge in pingdotgg/t3code#17028
* fix(server): a message sent during a rollback no longer undoes it by @t3dotgg in pingdotgg/t3code#17079
* fix(web): PR file stats ignore the hide-whitespace toggle by @flamboh in pingdotgg/t3code#16162
* perf(mobile): omit duplicated turn items from bounded thread snapshots by @juliusmarminge in pingdotgg/t3code#15385
* perf(mobile): pause elapsed-time timers on hidden thread screens by @juliusmarminge in pingdotgg/t3code#15397
* fix(mobile): pause hidden home thread list updates by @juliusmarminge in pingdotgg/t3code#15705
* fix(mobile): restore file viewer insets and glass header by @juliusmarminge in pingdotgg/t3code#17073
* fix(web): PR code toolbar no longer overlaps in narrow panels by @flamboh in pingdotgg/t3code#15561
* fix(web): PR commit menu no longer stretches across the window by @flamboh in pingdotgg/t3code#15560
* fix(web): command palette scrollbar no longer clipped at the top by @flamboh in pingdotgg/t3code#17035
* fix(web): Usage breadcrumb stays centered on small viewports by @flamboh in pingdotgg/t3code#15552
* fix(mobile): stop refreshing Git status on streamed thread updates by @juliusmarminge in pingdotgg/t3code#15893
* fix(mobile): skip move indexes for empty and single-thread sections by @juliusmarminge in pingdotgg/t3code#16115
* perf(mobile): reuse encoded rows in shell cache saves by @juliusmarminge in pingdotgg/t3code#16129
* perf(mobile): skip showcase subscriptions in normal builds by @juliusmarminge in pingdotgg/t3code#16131
* perf(mobile): remove unused Home project sorting by @juliusmarminge in pingdotgg/t3code#16177
* perf(mobile): reduce move-menu index allocations by @juliusmarminge in pingdotgg/t3code#16256
* perf(mobile): skip impossible thread-key lookups by @juliusmarminge in pingdotgg/t3code#16263
* fix(mobile): collect UI runtime garbage on iOS memory warnings by @juliusmarminge in pingdotgg/t3code#16296
* fix(mobile): stop Git sheet refresh loop by @juliusmarminge in pingdotgg/t3code#16305
* fix(mobile): refresh Git status after reconnect by @juliusmarminge in pingdotgg/t3code#16329
* fix(mobile): update the iOS Git header menu when status changes by @juliusmarminge in pingdotgg/t3code#16330
* perf(mobile): reuse the settled sort when settled rows are unchanged by @juliusmarminge in pingdotgg/t3code#16369
* perf(mobile): highlight source files in small batches that keep grammar state by @juliusmarminge in pingdotgg/t3code#16729
* feat(web): quote chips show what you said about the quote by @flamboh in pingdotgg/t3code#15703
* feat(web): projectless threads show their machine in the sidebar by @flamboh in pingdotgg/t3code#17022
* fix(lineage): keep agent effort and speed after completion by @Bil0000 in pingdotgg/t3code#16925
* fix(mobile): align diff scrolling with glass headers by @juliusmarminge in pingdotgg/t3code#17085
* fix(web): workspace page headers can no longer grow past the top-bar height by @maria-rcks in pingdotgg/t3code#17086
* feat(mobile): redesign the Add environment sheet by @juliusmarminge in pingdotgg/t3code#17092


**Full Changelog**: pingdotgg/t3code@v0.0.46-nightly.20261008.2801...v0.0.46-nightly.20261008.2813

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.46-nightly.20261008.2813
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant