Skip to content

fix(work-driver): the explore prompt asks for two different verdicts, and a complete spec can be parked as "underspecified" #397

Description

@randomm

What happened

/work 337 ran for 1m45s, spent 656k tokens, and produced a complete, evidence-grounded spec: a concrete intent, 2 deliverables with paths, 3 acceptance criteria, 7 pieces of executed evidence, and openQuestions: ["**None blocking** — mechanism is confirmed with executed evidence"].

It reported to the operator: "#337 does not say enough to build from. Add acceptance criteria or a concrete description and re-run."

The park was not right-for-the-wrong-reason. It was wrong — the resolver said proceed.

Verified root-cause chain

  1. inlineExplorePrompt asks for two different verdicts, both labelled LOAD-BEARING. The legacy ## Verdict block (work-driver-prompts-early.ts:34-55, doctrine at :54) and the feat(work-driver): resolve intent from any spec — ground it against code and world, then proceed, assume, or park #378 INTENT-VERDICT: block (:118-141, doctrine at :123). Worse, intentResolutionBlock(issues) is called unconditionally at :100 — never gated on intentResolutionEnabled() — so even PI_ENSEMBLE_INTENT=0 asks for a token the driver will not read.

  2. The resolver answered the legacy one. .pi/work-state/337/msk0oolc-z1j8y8-explore.txt:13-14 is ## Verdict / VERDICT: NEEDS_WORK — i.e. proceed. grep for INTENT-VERDICT or PARK-REASON in that reply returns nothing.

  3. The legacy token was never read. work-driver-explore.ts:220-221 reaches parseExploreVerdict only when parseNormalisedSpec returns undefined. A ## Spec block was present, so the intent path won and NEEDS_WORK was discarded.

  4. Both recorded values are parser defaults. work-driver-intent.ts:133-136park; :159parkReason: "underspecified". Missing and unrecognised are indistinguishable, and nothing records that the value was synthesised.

  5. All 7 evidence rows were downgraded to unverifiable. parseEvidence (work-driver-intent.ts:191-193) requires a bare confirmed; the resolver wrote — **confirmed**. Sibling parsers already tolerate bold (\** at :128 and :138) — this is the one place it was omitted. So explainCap found no confirmed evidence to display and shipped the boilerplate alone.

  6. reconcileVerdict never checks that a park is coherent. :212 returns immediately on any park, so park + 2 deliverables + 3 acceptance criteria + zero blocking questions sails through.

What to build

A1 — one protocol per dispatch. Add a 4th param intentEnabled: boolean to inlineExplorePrompt (:27-31); do NOT import intentResolutionEnabled there (prompts-early → intent → plan → prompts-early is an import cycle). Derive useLegacyVerdict = !intentEnabled || issues.length > 1 and make verdictBlock (:34-51), verdictDoctrine (:52-55) and intentResolutionBlock (:100) conditional on it. Filter empty strings before the join("\n") at :77-103.

Call site work-driver-explore.ts:164 passes intentResolutionEnabled() && issues.length === 1; the router guard at :219 gains the same && issues.length === 1.

That guard closes a latent second bug: today a multi-issue reply containing a ## Spec block returns at :244 having set neither activeIssues nor droppedIssues, so per-issue routing silently evaporates and every requested issue proceeds.

Keep parseExploreVerdict, parsePerIssueVerdicts and the legacy router branch at :249 — all three are live (the PI_ENSEMBLE_INTENT=0 hatch routes through them, and parsePerIssueVerdicts is the only per-issue router; parseNormalisedSpec yields one spec, not N).

A2 — parseEvidence tolerance at work-driver-intent.ts:191-193:

const tok = last.match(/^\**\s*(confirmed|contradicted)\b/)?.[1];

Anchored with a \** prefix: accepts — **confirmed** and — **confirmed** (distinct identity bypasses anti-recursion), still rejects prose like — I could not confirm this that an unanchored match would wrongly accept.

A3 — completeness predicate + scoped override. New exported specIsComplete(spec): intent non-empty AND ≥1 deliverable AND ≥1 acceptance criterion AND ≥1 confirmed evidence row AND no contradicted row AND no blocking open question (a question matching /^[\s*_]*(none|n/a)\b/i` is not blocking).

The some(confirmed) conjunct is what keeps "silence is not permission" true: a resolver that filled in the template without checking anything against the code has no confirmed row and still parks. It also couples A3 to A2 by design — without the bold tolerance, #337 scores zero confirmed rows and the predicate correctly returns false.

Apply at one site, in reconcileVerdict (:211-212). Scope the override to underspecified only — that is the single park reason a complete spec refutes on its face. contradicted-by-code, already-implemented, too-large and premise-unsound are all compatible with a complete spec and must still park.

The resulting verdict is proceed-with-assumptions, never proceed — the resolver never said proceed, the driver inferred it. Drop parkReason from the returned object (work-driver-explain.ts:47-48 and work-queue.ts:121-122 both read it) and append one synthetic assumption recording the override, so it surfaces where review happens via the existing renderAssumptions call at work-driver-commit.ts:140-141.

Acceptance criteria

  • A rendered explore prompt contains exactly one verdict marker, asserted for each of (single-issue, intent on), (multi-issue), (intent off).
  • CANARY extension/smoke-tests/fixtures/explore-replies/337.txt — the real captured reply — parses with all 7 evidence rows confirmed (today: all unverifiable).
  • CANARY the same fixture resolves to proceed-with-assumptions with parkReason === undefined (today: park / underspecified).
  • CANARY end-to-end: mocking the explore dispatch to return that fixture verbatim produces no intent-park cap-hit and fires the plan dispatch.
  • The override assumption reaches the PR body via renderAssumptions.
  • Bold, bold-with-parenthetical and bold-contradicted evidence forms all parse; — I could not confirm this stays unverifiable (pins the ^ anchor).
  • The same complete spec with PARK-REASON: too-large / already-implemented / premise-unsound / contradicted-by-code still parks — the guard that keeps the override narrow.
  • The existing "silence is not permission" assertions (test-intent-resolution.ts:191-198) stay green.
  • test-work-driver-pr6.ts, pr10-*, pr13.ts — which exercise the legacy router — stay green.
  • Quality gate green per AGENTS.md §1; docs updated.

Why the existing tests missed it

Every evidence fixture in test-intent-resolution.ts uses a bare confirmed, and every one carries an INTENT-VERDICT:. They were written to match the regexes. test-grouping-real-issues.ts:5-7 already documents this exact pathology for the grouping rules; the fix is the same — a fixture captured verbatim from a real reply.

This work must ship as its own separate PR, independent of any other open issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions