feat(#378): resolve intent from any spec, then proceed, assume, or park - #379
Merged
Conversation
/work assumed an issue told it what to build. Real backlogs do not honour
that — issues are hand-written, terse, imported from another project, or
simply wrong. Externally, 38.3% of real GitHub issues are underspecified
(SWE-bench Verified, 93 annotators over 1,699 samples), and 41.77% of
multi-agent failures are specification-level (MAST), the largest single
category. Underspecification is the modal case, not an edge case.
Two structural problems this replaces:
1. A MISSING verdict meant "build it" — work-driver-plan.ts defaulted an
absent token to NEEDS_WORK, so an agent that simply forgot the marker
got code written. Silence was permission.
2. Nothing asked whether the issue was TRUE. No step checked whether the
named symbols exist, whether the described behaviour matches the code,
or whether the work was already done. A confidently-wrong bug report
got built.
The explore step now resolves intent from whatever body it is handed and
grounds it — against the code (do these symbols exist? does the behaviour
match? is it already done?) and against the world (does that API exist
with that shape?). It returns proceed / proceed-with-assumptions / park,
with machine-readable park reasons: underspecified, contradicted-by-code,
already-implemented, too-large, premise-unsound.
A park halts at `intent-park` BEFORE plan or branch, so refusing costs one
explore dispatch rather than a whole cycle ending in a bad PR. And a
missing or unreadable verdict now parks — the inversion of the old
default.
Evidence overrides the resolver's own verdict. An LLM can claim `proceed`
while recording that the issue's central claim is contradicted by the
code; when it does, the contradiction wins. That is the highest-value
signal this step produces and ignoring it is precisely how a wrong bug
report gets built.
`proceed-with-assumptions` writes every assumption into the PR body. The
verdict is only honest if review can see what was assumed; buried in a
state file it may as well not exist.
The resolver runs in the `explore` role, which role-tools.ts already gates
with --exclude-tools write,edit,multiedit (#238). That is load-bearing
rather than incidental: "Ask or Assume?" finds an agent holding edit tools
rationalises ambiguity away, because building is cheaper than asking.
Also fixes the #290 false positive I shipped: countEnumeratedFindings
counts top-level `- [ ]`, which in a /plan issue is exclusive to
`## Acceptance criteria`. Measured on real bodies — #287→7, #288→6,
#289→7, #366→6 — every count was test assertions, with ZERO of #287's five
actual deliverables (`**A.**`–`**E.**`) seen, since bolded letters are
invisible to it. So a correctly-planned single-workstream issue triggered
a corrective re-dispatch essentially every time. Decomposition is now
counted from the resolved deliverables.
Format markers stay a fast path, never a contract: no change to /plan, and
a reply with no `## Spec` block falls through to the legacy verdict router
rather than parking every cycle on drift.
74/74 offline smoke tests, tsc + biome clean.
Fixes #378
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #378. The core of Wave 1.
The problem
/workassumed the issue told it what to build. Real backlogs don't honour that — issues are hand-written, terse, imported from elsewhere, or simply wrong. 38.3% of real GitHub issues are underspecified (SWE-bench Verified, 93 annotators, 1,699 samples) and 41.77% of multi-agent failures are specification-level (MAST, the largest single category). Underspecification is the modal case.Two structural holes:
work-driver-plan.tsdefaulted an absent token toNEEDS_WORK. An agent that simply forgot the marker got code written. Silence was permission.What it does now
The explore step resolves intent from whatever body it is handed and grounds it:
Then it decides:
proceedproceed-with-assumptionsparkintent-parkwith a machine-readable reasonPark reasons —
underspecified,contradicted-by-code,already-implemented,too-large,premise-unsound— each carry their own operator action, not the generic "inspect the state file and--restart" (which for an unresolvable issue just reproduces the same park).A park halts before plan or branch, so refusing costs one explore dispatch rather than a whole cycle ending in a bad PR. Verified:
nextStepafterintent-parkreturnshandoff, so no code-writing step can run.A missing or unreadable verdict now parks. That is the inversion.
Two design points worth arguing
Evidence overrides the resolver's own verdict. An LLM can claim
proceedwhile recording that the issue's central claim is contradicted by the code. When it does, the contradiction wins. That contradiction is the highest-value signal this step produces, and ignoring it is precisely how a wrong bug report gets built. Tested directly.The resolver structurally cannot build. It runs in the
explorerole, whichrole-tools.tsalready gates with--exclude-tools write,edit,multiedit(#238). Load-bearing rather than incidental: "Ask or Assume?" (69.4% on an underspecified SWE-bench variant) finds that an agent holding edit tools rationalises ambiguity away, because building is cheaper than asking.Also fixes the #290 false positive I shipped yesterday
countEnumeratedFindingscounts top-level- [ ], and in a/planissue checkboxes are exclusive to## Acceptance criteria. Measured on real bodies — #287→7, #288→6, #289→7, #366→6 — every count was test assertions, with zero of #287's five actual deliverables (**A.**–**E.**) seen, because bolded letters are invisible to it. A correctly-planned single-workstream issue therefore triggered a corrective re-dispatch essentially every time. Decomposition is now counted from the resolveddeliverables[].What this does NOT do
No change to
/plan. Its job is to write the best spec it can;/work's job is to cope with whatever it is handed. Format markers stay a fast path, never a contract — and a reply with no## Specblock falls through to the legacy verdict router rather than parking every cycle on drift.Verification
Full §1 gate: build,
tsc --noEmit,bun run check, 74/74 offline smoke tests. All touched files under the 500-line cap.test-intent-resolution.ts— 30 assertions across the real range of spec quality: a well-formed spec; a one-line human bug report ("login is broken on mobile") that must park rather than invent deliverables; an issue contradicted by the code; already-implemented; too-large; a missing verdict; an unparseable verdict; and a reply with no## Specblock at all.The assertions that matter most are the negative ones — no deliverables are invented from a one-liner, and a
proceedclaimed alongside contradicting evidence is overridden.One implementation note:
sliceMarkdownSectionmatches exactly##and cannot reach###subsections. I added a local slicer rather than generalising the shared helper —parseWorkstreamsdepends on its current terminator behaviour, and widening a shared parser to serve one new caller is how subtle regressions get in.Escape hatch
PI_ENSEMBLE_INTENT=0. Docs: README knob, atroubleshooting.mdsection explaining each park reason, AGENTS.md §7.