Skip to content

feat(#378): resolve intent from any spec, then proceed, assume, or park - #379

Merged
randomm merged 1 commit into
mainfrom
feature/issue-378-intent-resolution
Aug 7, 2026
Merged

feat(#378): resolve intent from any spec, then proceed, assume, or park#379
randomm merged 1 commit into
mainfrom
feature/issue-378-intent-resolution

Conversation

@randomm

@randomm randomm commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Closes #378. The core of Wave 1.

The problem

/work assumed the issue told it what to build. Real backlogs don't honour that — issues are hand-written, terse, imported from elsewhere, or simply wrong. 38.3% of real GitHub issues are underspecified (SWE-bench Verified, 93 annotators, 1,699 samples) and 41.77% of multi-agent failures are specification-level (MAST, the largest single category). Underspecification is the modal case.

Two structural holes:

  1. A missing verdict meant "build it." work-driver-plan.ts defaulted an absent token to NEEDS_WORK. An agent that simply forgot the marker got code written. Silence was permission.
  2. Nothing asked whether the issue was TRUE. No step checked whether the named symbols exist, whether the described behaviour matches the code, or whether the work was already done. A confidently-wrong bug report got built.

What it does now

The explore step resolves intent from whatever body it is handed and grounds it:

  • against the code — do these symbols exist? does the behaviour match? is it already implemented?
  • against the world — does that third-party API exist with that shape?

Then it decides:

Verdict Action
proceed continue
proceed-with-assumptions continue, every assumption written into the PR body
park write no code; halt at intent-park with a machine-readable reason

Park reasons — underspecified, contradicted-by-code, already-implemented, too-large, premise-unsound — each carry their own operator action, not the generic "inspect the state file and --restart" (which for an unresolvable issue just reproduces the same park).

A park halts before plan or branch, so refusing costs one explore dispatch rather than a whole cycle ending in a bad PR. Verified: nextStep after intent-park returns handoff, so no code-writing step can run.

A missing or unreadable verdict now parks. That is the inversion.

Two design points worth arguing

Evidence overrides the resolver's own verdict. An LLM can claim proceed while recording that the issue's central claim is contradicted by the code. When it does, the contradiction wins. That contradiction is the highest-value signal this step produces, and ignoring it is precisely how a wrong bug report gets built. Tested directly.

The resolver structurally cannot build. It runs in the explore role, which role-tools.ts already gates with --exclude-tools write,edit,multiedit (#238). Load-bearing rather than incidental: "Ask or Assume?" (69.4% on an underspecified SWE-bench variant) finds that an agent holding edit tools rationalises ambiguity away, because building is cheaper than asking.

Also fixes the #290 false positive I shipped yesterday

countEnumeratedFindings counts top-level - [ ], and in a /plan issue checkboxes are exclusive to ## Acceptance criteria. Measured on real bodies — #287→7, #288→6, #289→7, #366→6 — every count was test assertions, with zero of #287's five actual deliverables (**A.****E.**) seen, because bolded letters are invisible to it. A correctly-planned single-workstream issue therefore triggered a corrective re-dispatch essentially every time. Decomposition is now counted from the resolved deliverables[].

What this does NOT do

No change to /plan. Its job is to write the best spec it can; /work's job is to cope with whatever it is handed. Format markers stay a fast path, never a contract — and a reply with no ## Spec block falls through to the legacy verdict router rather than parking every cycle on drift.

Verification

Full §1 gate: build, tsc --noEmit, bun run check, 74/74 offline smoke tests. All touched files under the 500-line cap.

test-intent-resolution.ts — 30 assertions across the real range of spec quality: a well-formed spec; a one-line human bug report ("login is broken on mobile") that must park rather than invent deliverables; an issue contradicted by the code; already-implemented; too-large; a missing verdict; an unparseable verdict; and a reply with no ## Spec block at all.

The assertions that matter most are the negative ones — no deliverables are invented from a one-liner, and a proceed claimed alongside contradicting evidence is overridden.

One implementation note: sliceMarkdownSection matches exactly ## and cannot reach ### subsections. I added a local slicer rather than generalising the shared helper — parseWorkstreams depends on its current terminator behaviour, and widening a shared parser to serve one new caller is how subtle regressions get in.

Escape hatch PI_ENSEMBLE_INTENT=0. Docs: README knob, a troubleshooting.md section explaining each park reason, AGENTS.md §7.

/work assumed an issue told it what to build. Real backlogs do not honour
that — issues are hand-written, terse, imported from another project, or
simply wrong. Externally, 38.3% of real GitHub issues are underspecified
(SWE-bench Verified, 93 annotators over 1,699 samples), and 41.77% of
multi-agent failures are specification-level (MAST), the largest single
category. Underspecification is the modal case, not an edge case.

Two structural problems this replaces:

  1. A MISSING verdict meant "build it" — work-driver-plan.ts defaulted an
     absent token to NEEDS_WORK, so an agent that simply forgot the marker
     got code written. Silence was permission.
  2. Nothing asked whether the issue was TRUE. No step checked whether the
     named symbols exist, whether the described behaviour matches the code,
     or whether the work was already done. A confidently-wrong bug report
     got built.

The explore step now resolves intent from whatever body it is handed and
grounds it — against the code (do these symbols exist? does the behaviour
match? is it already done?) and against the world (does that API exist
with that shape?). It returns proceed / proceed-with-assumptions / park,
with machine-readable park reasons: underspecified, contradicted-by-code,
already-implemented, too-large, premise-unsound.

A park halts at `intent-park` BEFORE plan or branch, so refusing costs one
explore dispatch rather than a whole cycle ending in a bad PR. And a
missing or unreadable verdict now parks — the inversion of the old
default.

Evidence overrides the resolver's own verdict. An LLM can claim `proceed`
while recording that the issue's central claim is contradicted by the
code; when it does, the contradiction wins. That is the highest-value
signal this step produces and ignoring it is precisely how a wrong bug
report gets built.

`proceed-with-assumptions` writes every assumption into the PR body. The
verdict is only honest if review can see what was assumed; buried in a
state file it may as well not exist.

The resolver runs in the `explore` role, which role-tools.ts already gates
with --exclude-tools write,edit,multiedit (#238). That is load-bearing
rather than incidental: "Ask or Assume?" finds an agent holding edit tools
rationalises ambiguity away, because building is cheaper than asking.

Also fixes the #290 false positive I shipped: countEnumeratedFindings
counts top-level `- [ ]`, which in a /plan issue is exclusive to
`## Acceptance criteria`. Measured on real bodies — #287→7, #288→6,
#289→7, #366→6 — every count was test assertions, with ZERO of #287's five
actual deliverables (`**A.**`–`**E.**`) seen, since bolded letters are
invisible to it. So a correctly-planned single-workstream issue triggered
a corrective re-dispatch essentially every time. Decomposition is now
counted from the resolved deliverables.

Format markers stay a fast path, never a contract: no change to /plan, and
a reply with no `## Spec` block falls through to the legacy verdict router
rather than parking every cycle on drift.

74/74 offline smoke tests, tsc + biome clean.

Fixes #378
@randomm
randomm merged commit db806b1 into main Aug 7, 2026
1 check passed
@randomm
randomm deleted the feature/issue-378-intent-resolution branch August 7, 2026 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(work-driver): resolve intent from any spec — ground it against code and world, then proceed, assume, or park

1 participant