Skip to content

ref(chat): Simplify durable conversation execution - #1344

Merged
dcramer merged 48 commits into
mainfrom
ref/simple-conversation-execution
Aug 9, 2026
Merged

ref(chat): Simplify durable conversation execution#1344
dcramer merged 48 commits into
mainfrom
ref/simple-conversation-execution

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Use one mailbox, one conversation lease, and one worker for durable conversation work. SQL conversation events keep the history. Redis stores temporary execution state in new v2 keys.

Reliability rules

A turn can continue after a timeout, retry, or yield only when its committed boundary changes. If the turn parks at the same boundary again, the worker stops the turn and records the error.

A process can stop while a turn runs. The next worker stops that turn. The user can start new work. Committed SQL history remains.

Queue work uses the conversation lease. OAuth can run outside the queue and uses the thread lock. One cursor lock controls each cursor write. A stale writer cannot change SQL or Redis after it loses the lock.

New Redis keys

This release reads and writes only junior:conversation:v2:* and junior:turn_cursor:v2:*. It does not read, move, or write old Redis state. Old mailbox, lease, and turn-cursor state can be lost. SQL schema and committed history do not change. Rollback support is out of scope.

Integration tests

Production and the durable-queue tests use the same createConversationWork composition. The tests replace agent behavior and Slack HTTP. They use the real StateAdapter with memory storage. They use an in-memory queue that implements the one-method queue interface.

The tests cover these product rules:

  • One accepted turn commits, sends one reply, and drains.
  • A new instruction can steer an active turn.
  • An authorization request pauses a turn and does not retry it.
  • An error before input commit retries and does not send a duplicate reply.
  • A repeated timeout boundary stops the turn.
  • A lost worker stops its running turn and keeps committed history.
  • Repeated errors stop at the retry limit and send at most one fallback reply.

Refs JUNIOR-62

Replace the five near-identical persist* session helpers with
saveTurnCheckpoint(mode: running | paused | completed | failed). Resume
uses that API only, adopts the committed boundary on continue, and fails
closed when a slice parks again at the same resumed boundary.

Append history checks durable message identity so Pi in-place envelope
mutations no longer look like branches. SQL event history is unchanged;
Redis remains a thin resume cursor.

Co-Authored-By: David Cramer <david@sentry.io>
@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
junior-docs Ready Ready Preview Aug 9, 2026 6:53pm

Request Review

Resume now imports saveTurnCheckpoint; the sandbox component mock still
only exported the old persist* helpers and failed the suite.

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a useful step toward the durable-queue model, but it does not yet land the containment you described for that system. Durable identity + a single write function are real improvements; the dual execution machines, old session-record surface, and compatibility layer remain the main problem.

Please treat this as a containment cut, not a rename of persist*:

  1. Contain the durable queue path. The PR still leaves conversation work (task-execution) and turn-session resume as two machines that both answer “is this alive / can we continue?”. File names and public API still say turn-session-record / SessionRecord / sessionId, while callers also say turnId. For the core queue, a maintainer should be able to open one small module family and answer: mailbox, lease, checkpoint cursor, worker loop.
  2. Hard-cut the old write API. Production still imports the five persist* / completeDeliveredTurn shims. Policy requires either a hard cutover or TODO(vX.Y.Z) on compatibility paths. You already approved abandoning in-flight turns, so cut the callers and delete the shims in this PR (or the immediate follow-up before merge).
  3. Shrink the public write shape. TurnCheckpointBase is still a bag of routing, metrics, provenance, and delivery fields. Keep the public API to the few fields the lifecycle needs; pass SQL dual-write/live routing through a narrower internal path if needed.
  4. Use one vocabulary. New code should use turnId (terminology), one pause/status word (paused vs stored awaiting_resume needs one mapping, not two public dialects), and stop introducing SessionRecord / loadTurnSessionRecord names next to saveTurnCheckpoint.

The durable-identity change and resume baseline are the right local fixes for JUNIOR-62. I would not merge until the surface is small enough that “checkpoint” is obvious without reading eight files.

Comment thread packages/junior/src/chat/services/turn-session-record.ts Outdated
Comment thread packages/junior/src/chat/services/turn-session-record.ts Outdated
Comment thread packages/junior/src/chat/services/turn-session-record.ts Outdated
Comment thread packages/junior/src/chat/agent/resume.ts Outdated
Comment thread packages/junior/src/chat/services/turn-session-record.ts Outdated
Delete the dual turn-session-record surface and one-hop session wrapper.
One API lives with the durable queue: loadTurnCheckpoint / saveTurnCheckpoint.
Callers and tests use turnId; shims are gone. SQL history untouched.

Co-Authored-By: David Cramer <david@sentry.io>

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hard cut to task-execution/checkpoint addresses the earlier API/ownership feedback. There are still two things to fix before this lands:

  • The no-progress path needs behavior coverage that resumes an awaiting_resume checkpoint and proves an unchanged timeout/retry boundary becomes terminal. The current resume test only covers auth persistence, so the main fix for JUNIOR-62 is untested.
  • CI is currently red for PR-owned changes: the renamed checkpoint test exceeds the file-length limit while the exception still names the deleted file, and the cooperative-yield assertion expects the old error text. Please fix both rather than merging with the required check failing.

Comment thread packages/junior/src/chat/agent/resume.ts Outdated
Runtime status is paused. Checkpoint + continue live under task-execution.
SQL free-text/enums keep awaiting_resume via edge mapping only. No SQL
schema change. In-flight resume records may drop on deploy.

Co-Authored-By: David Cramer <david@sentry.io>
@sentry-junior sentry-junior Bot changed the title ref(chat): Collapse turn checkpoints to one write path ref(chat): Contain turn checkpoints in task-execution Aug 8, 2026
Only treat a running checkpoint as progress when the persisted
boundary differs from the resumed one, and cover timeout plus
same-boundary running writes in the no-progress regression.
@sentry-junior sentry-junior Bot changed the title ref(chat): Contain turn checkpoints in task-execution ref(chat): Contain turn checkpoints in the durable queue Aug 8, 2026
Queue continue already holds the conversation work lease. Skip the
extra thread lock and the ResumeTurnBusy retry loop so continue is one
owner, not two. OAuth out-of-band resumes still lock. Continue reads
go through loadTurnCheckpoint.

Co-Authored-By: David Cramer <david@sentry.io>
Move continue-run and turn-cursor storage under the durable queue
folder. Checkpoint is the only external gate; outside callers no
longer import turn storage or the old runtime continue runner path.

Co-Authored-By: David Cramer <david@sentry.io>
@sentry-junior sentry-junior Bot changed the title ref(chat): Contain turn checkpoints in the durable queue ref(chat): Contain the durable queue under task-execution Aug 9, 2026
@sentry-junior sentry-junior Bot changed the title ref(chat): Contain the durable queue under task-execution ref(chat): Simplify durable conversation execution Aug 9, 2026

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial pass on complexity / terminology / dual surfaces. The hard-cut and no-progress tests moved the right direction, but this still reads like a rename of the old turn-session machine more than a thinner execution model.

Please fix these before merge:

  1. Containment is fake. checkpoint.ts claims to be the only external gate, then re-exports getTurnRecord / failTurnRecord / summaries, and production still imports turn-cursor directly (oauth-callback, reply-executor, agent-dispatch/work, agent-invocations/work, continue-run). One public module. Internal storage stays private.
  2. One turn identity. Runtime still juggles turnId args against sessionId fields / sessionRecord locals / app.ai.resume_session_id. TurnRecord.sessionId should be turnId at the TypeScript boundary; keep the Redis key mapping in one private place.
  3. turn-cursor.ts is still the old bag (~1100 LOC + exception). A "thin cursor" should not own SQL dual-write, runtime-context splice, history-version follow, and recovery index TTL policy in one file. Shrink or split by real concern — do not land another rename with a file-length exception.
  4. Yield is outside the no-progress fail-closed path. translateSuspension only checks same-boundary spin for timeout/retry. Cooperative yield can re-park the identical boundary forever while shouldYield() stays true. Apply the same fail-closed rule (or an explicit slice/progress counter) to yield.
  5. Checkpoint write shape is still a grab-bag. TurnCheckpointWrite + sharedWrite optional-merge every routing/metrics/provenance field. Public write should be lifecycle-only; SQL dual-write metadata should not dominate the API.

I would not ship this as "simplified conversation execution" until a maintainer can open one small module family and not also keep the old session vocabulary in their head.

getTurnRecord,
listTurnSummaries,
recordTurnSummary,
};

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This re-export list breaks the claim two lines above that turn-cursor.ts is internal. Callers already import storage helpers from here and from ./turn-cursor directly. Delete these re-exports and force every production caller through the real checkpoint lifecycle API (load / save / maybe explicit fail+abandon wrappers that take turnId). If a read helper is truly needed outside this folder, give it a checkpoint-shaped name and stop exporting the storage record API.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. Production code outside task-execution no longer imports turn-cursor. checkpoint.ts is the external gate. paused-turn.ts is part of the same execution owner and uses the private store directly.

Comment thread packages/junior/src/chat/task-execution/checkpoint.ts Outdated
turnContexts?: PluginTurnContext[];
durationMs?: number;
usage?: AgentTurnUsage;
errorMessage?: string;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TurnCheckpointWrite is still the old optional wall (destination/source/actor/skills/provenance/usage/dispatch receipts). That is not a lifecycle API. Split to required progress fields only; pass dual-write routing through a narrow internal path if SQL still needs it this PR.

).message,
sliceId: args.sliceId,
state: "failed",
});

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

savePaused can return a record with state: "failed" on the slice-limit path. Callers then branch on record.state === "paused" vs throw. Prefer throwing TurnSliceLimitExceededError here (or returning a discriminated result) so "saved pause" cannot mean "terminal failure".

Comment thread packages/junior/src/chat/agent/resume.ts Outdated
Comment thread packages/junior/src/chat/task-execution/turn-cursor.ts Outdated
Comment thread packages/junior/src/chat/task-execution/continue-run.ts Outdated
Co-Authored-By: David Cramer <david@sentry.io>
Co-Authored-By: David Cramer <david@sentry.io>

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can follow the local mechanics, but I still cannot state the intended contract confidently enough to maintain this.

My current reading is: the conversation lease owns normal liveness; a checkpoint points into SQL history; paused timeout/retry/yield work is resumed by the worker; repeating a timeout/retry boundary fails the turn; a running checkpoint with no other owner is instead recovered; OAuth remains an out-of-band exception.

Please clarify which of these is the actual goal:

  • Is this a narrow fix for JUNIOR-62, or a migration to a new durable conversation execution model? The title/body say “simplify,” but the diff also defines worker-death recovery, history-replacement following, legacy-state normalization, summary-index retention, and OAuth exceptions across 72 files.
  • After a deploy or hard worker death, is preserving an in-flight running turn a required contract, or may it fail closed? Earlier PR text explicitly allowed in-flight turns to fail; current code adds recovery, while the current body no longer states the policy. This decision changes whether most of recoverStrandedRunningSession and its fallbacks belong here.
  • Why is cooperative yield allowed to repeat the same boundary when timeout/retry are not? If yield is expected to make no model progress while handing off a lease, say that and identify what guarantees it eventually stops; otherwise it belongs under the same no-progress rule.

Please put the chosen state machine and failure policy in task-execution/README.md in plain terms: owner, legal states/transitions, what survives process death, and which exceptions are intentional. Right now those answers have to be inferred from several files and old review history.

Comment thread packages/junior/tests/integration/durable-queue.test.ts Outdated

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still does not meet the stated one-queue/one-owner design, and two retry paths remain unsafe. Please address these before merge:

  • durable-queue.test.ts does not exercise the production continue path for JUNIOR-62. Its incident case injects a custom worker callback and manually calls createResumeState; it bypasses resumeAwaitingSlackContinuation / continueSlackAgentRun, resumeSlackTurn, production routing, and the worker's actual paused-turn selection. It proves the local no-progress helper, not queue → production worker → checkpoint → redelivery. Keep the small harness, but make the incident scenario invoke the real worker entry with only the agent/external boundary injected.
  • Cooperative yield bypasses the no-progress guard in agent/resume.ts. A resumed run that immediately yields at the same boundary can persist paused forever because yield neither advances the slice nor compares against resumedBoundaryKey. Apply the same fail-closed progress rule to yield and cover it in the high-level suite.
  • savePaused can return a terminal failed record when the slice limit is reached. parkForAuth treats every truthy result as a successful auth pause and returns awaiting_auth, so a failed turn can be exposed as resumable. Make a pause operation either return a paused checkpoint or throw/return an explicit terminal result; callers must not infer success from truthiness.
  • checkpoint.ts is not a small lifecycle boundary. TurnCheckpointWrite still exposes routing, actor, provenance, metrics, dispatch, and delivery fields, and the module re-exports raw record/summary storage operations. Seventeen production files import this surface, while continue-run.ts bypasses it and imports turn-cursor directly. Reduce the public API to lifecycle operations with small projections; keep SQL metadata and storage records private.
  • The claimed terminology/containment cut is incomplete. Public TurnRecord/TurnSummary still expose sessionId; continue-run.ts uses sessionRecord; the supposedly thin cursor remains a 1,098-line renamed session implementation with a file-length exception; and store.ts is a 536-line mirror with 27 forwards into state.ts. This is containment by folder, not the understandable durable queue requested. Normalize turnId at the storage boundary and remove the forwarding/duplicate surfaces rather than documenting them as the final design.

What is solid: SQL schema/history are unchanged, durable prefix identity correctly ignores Pi's volatile assistant-envelope mutations while rejecting true branches, terminal writes retry, and CI is green. The PR is net -252 production lines, but the architecture and integration proof are not yet at the bar described in the PR and thread.

@sentry-junior

sentry-junior Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Quality pass at c8a5548:

  • Removed the lower-level timeout/yield/retry no-progress matrix. The durable-queue integration suite now owns the repeated-boundary behavior and asserts the terminal turn invariant through production composition.
  • Kept lower-level tests only for distinct contracts: auth error propagation, queue transport mapping, StateAdapter behavior, and lock fencing/lease semantics.
  • Removed seven unused public type/constant exports from the new execution surface.
  • Updated remaining turn-cursor comments to use the current vocabulary.
  • Net change: 54 fewer lines.

Verified: 26 targeted tests, typecheck, lint/dependency rules, formatting, file-length, test-architecture, and diff checks.

Co-Authored-By: David Cramer <david@sentry.io>
Co-Authored-By: David Cramer <david@sentry.io>
Comment thread packages/junior/src/chat/task-execution/paused-turn.ts Outdated
Co-Authored-By: David Cramer <david@sentry.io>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7a0b5ec. Configure here.

Comment thread packages/junior/src/chat/task-execution/paused-turn.ts
Co-Authored-By: David Cramer <david@sentry.io>
Co-Authored-By: David Cramer <david@sentry.io>
Co-Authored-By: David Cramer <david@sentry.io>
Co-Authored-By: David Cramer <david@sentry.io>
@dcramer
dcramer merged commit b58da4c into main Aug 9, 2026
42 checks passed
@dcramer
dcramer deleted the ref/simple-conversation-execution branch August 9, 2026 19:02
dcramer added a commit that referenced this pull request Aug 9, 2026
Supersedes #1328 after #1344 moved `turn-session*` → turn-cursor /
checkpoint.

Stop threading `modelId` / skills / `reasoningLevel` through turn-cursor
writes. Those fields were never restored from storage, so the write path
was pretending they were checkpoint metadata.

**Ownership (unchanged)**
- checkpoint = resume boundary only (status, history boundary, slice)
- profile + reasoning = turn route / history replacement
- concrete model id = resolve at execution from profile config
- skills = derive from history + catalog
- channel + duration/usage = SQL conversation row

**What changed**
- Drop the three dead fields from `TurnRecord`, `upsertTurnRecord`, and
checkpoint write args
- Rename `runtimeMetadata` → `runtimeMetrics` for SQL-backed
channel/metrics only
- Remove the dead resume `existingSessionRecord.reasoningLevel` fallback
- Explicit post-handoff contract: profile config, else inherited old
route
- No recovery-index policy changes

Checked: `tsc --noEmit` (@sentry/junior + @sentry/junior-evals); focused
vitest checkpoint + agent-resume + paused-turn **45/45**.

Refs #1267

<!-- junior-request-attribution:start -->
Requested by **U039RR91S**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AG6MCDB51U%3A1786211808.864869 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AG6MCDB51U%3A1786211808.864869)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AG6MCDB51U%3A1786211808.864869/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: David Cramer <david@sentry.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: high PR risk score: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant