Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,7 @@ All optional. Defaults are reasonable for typical use.
| `PI_ENSEMBLE_PARALLEL_WORK` | `1` | Set to `0` to force strictly sequential group execution regardless of `PI_ENSEMBLE_PARALLEL_GROUPS`. |
| `PI_ENSEMBLE_MAX_WORKSTREAMS` | `6` | Ceiling on workstreams per cycle. Excess folds into the last workstream (paths unioned, fold recorded) rather than being dropped. Each workstream is a worktree and a developer child, so this bounds fanout. |
| `PI_ENSEMBLE_PLAN_QUALITY` | `1` | Set to `0` to disable the plan-quality re-dispatch (under-decomposed plan, or a workstream with no declared paths). The prompt doctrine stays either way. |
| `PI_ENSEMBLE_INTENT` | `1` | Set to `0` to disable intent resolution and restore the single-token explore verdict. Default ON — the driver resolves what an issue is actually asking for from **any** body (a full spec, a paragraph, a one-line bug report), grounds it against the code and the world, and then decides: proceed, proceed-with-assumptions, or **park** without writing code. A missing or unreadable verdict parks; silence is never permission. |

**Sandbox mode (`pi-ensemble` wrapper):**

Expand Down
28 changes: 28 additions & 0 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,34 @@ The "removed X" classes only fire when the token is genuinely absent from the hu

Escape hatch: `PI_ENSEMBLE_WIDENING_SCAN=0`.

### Intent resolution: why a cycle refused to write code

Before planning, `/work` resolves what the issue is actually asking for — from whatever body it was given — and checks whether that is *true* against the code and against the world. It then decides one of three things:

| Verdict | Meaning |
|---|---|
| `proceed` | Intent clear and grounded |
| `proceed-with-assumptions` | Gaps existed, each filled with a defensible default. **Every assumption appears in the PR body** so review sees exactly what was assumed |
| `park` | No code written. The cycle halts at `intent-park` before plan or branch runs |

Park reasons, each with its own operator action:

| Reason | What it means |
|---|---|
| `underspecified` | The issue does not say enough to build from |
| `contradicted-by-code` | Its central claim disagrees with the code as it actually is — usually a stale issue, or one already fixed |
| `already-implemented` | The work appears done |
| `too-large` | Not executable as one cycle; split it |
| `premise-unsound` | It rests on an API or behaviour that could not be substantiated |

**A missing or unreadable verdict parks.** Before this, an absent token defaulted to "build it" — silence was treated as permission. Parking early is cheap: it costs one explore dispatch rather than a whole cycle ending in a bad PR.

The resolver runs in the `explore` role, which is structurally denied `write`/`edit`/`multiedit`. That is deliberate — an agent holding edit tools rationalises ambiguity away, because building is cheaper than asking.

Grouping markers (`Split:`, `Depends-on:`, subsystem tags) remain a **fast path**, never a requirement. Hand-written and imported issues resolve the same way.

Escape hatch: `PI_ENSEMBLE_INTENT=0`.

### Parallel group execution

`/work N M P …` runs up to `PI_ENSEMBLE_PARALLEL_GROUPS` (default **3**) groups concurrently. Each group develops in its own `.worktrees/` tree, so the only shared resource is the repo root, and every operation that touches it — branch creation, patch integration, commit, push, `gh pr create`, the verify gates, lens-fix re-integration, `restoreCheckout`, worktree teardown — runs under a single integration lock. That lock is an in-process promise chain plus an `O_EXCL` lockfile under `.git/`, so a second `/work` invocation or a second Pi process on the same clone is also serialised.
Expand Down
270 changes: 270 additions & 0 deletions extension/smoke-tests/test-intent-resolution.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,270 @@
#!/usr/bin/env bun
/**
* #378 — intent resolution across the real range of spec quality.
*
* `/work` used to assume the issue told it what to build, and a MISSING
* verdict meant "build it" — silence was permission. Externally, 38.3% of
* real GitHub issues are underspecified (SWE-bench Verified) and 41.77% of
* multi-agent failures are specification-level (MAST), so underspecification
* is the modal case rather than an edge case.
*
* The fixtures below are what a real backlog actually contains: a full spec, a
* one-line human bug report, an issue contradicted by the code, an issue
* already implemented, and a reply that drifted off-format entirely.
*/

import {
type NormalisedSpec,
explainPark,
intentResolutionEnabled,
parkAction,
parseNormalisedSpec,
reconcileVerdict,
renderAssumptions,
} from "../src/work-driver-intent.ts";

let exit = 0;
function assert(cond: boolean, msg: string) {
if (cond) {
console.log(`✓ ${msg}`);
} else {
console.error(`✗ ${msg}`);
exit = 1;
}
}

const resolve = (t: string) => {
const p = parseNormalisedSpec(t);
return p ? reconcileVerdict(p) : undefined;
};

// ------------------------------------------------ a well-formed spec

const WELL_FORMED = `
INTENT-VERDICT: proceed

## Spec

### Intent
Make the branch step refuse to rebuild an issue that already has an open PR.

### Deliverables
- preflight: query open PRs and match on the issue number [paths: src/work-driver-pr-preflight.ts, src/work-driver-branch-develop.ts]
- handoff: render the existing PR in the handoff body [paths: src/work-driver-handoff-message.ts]

### Acceptance criteria
- A fresh cycle with no open PR is unaffected
- A cycle with an open PR halts before any dispatch

### Out of scope
- Adopting the existing branch

### Evidence
- gh pr create is called unconditionally — src/work-driver-commit.ts:212 — confirmed

## Rationale
The issue names concrete files that exist and the behaviour matches the code.
`;

{
const s = resolve(WELL_FORMED);
assert(s?.verdict === "proceed", "a well-formed spec resolves to proceed");
assert(s?.deliverables.length === 2, "deliverables are derived (2)");
assert(
s?.deliverables[0]?.paths.includes("src/work-driver-pr-preflight.ts") === true,
"deliverable paths are parsed",
);
assert(s?.deliverables[0]?.id === "preflight", "deliverable ids are parsed");
assert(
s?.acceptanceCriteria.length === 2,
"acceptance criteria are kept SEPARATE from deliverables",
);
assert(s?.outOfScope.length === 1, "out-of-scope is captured");
assert(s?.evidence[0]?.verdict === "confirmed", "evidence verdicts are parsed");
}

// ------------------------------------- a one-line human bug report

{
// The shape that matters most: no structure at all. The resolver must park
// rather than invent deliverables.
const s = resolve(`
INTENT-VERDICT: park
PARK-REASON: underspecified

## Spec

### Intent
Unclear — "login is broken on mobile" names no component, platform, or expected behaviour.

### Open questions
- Which login flow, and on which platform?

## Rationale
Nothing in the repo obviously corresponds to this. Building from a guess here
would produce a confident change to the wrong thing.
`);
assert(s?.verdict === "park", "a one-line human bug report parks rather than guessing");
assert(s?.parkReason === "underspecified", "park reason is underspecified");
assert(s?.deliverables.length === 0, "no deliverables are invented");
assert(
explainPark("underspecified", 42).includes("#42"),
"the operator explanation names the issue",
);
assert(
/add acceptance criteria/.test(parkAction("underspecified", 42)),
"the human action is specific, not 'inspect the state file'",
);
}

// ---------------------------------- contradicted by the code

{
// The highest-value signal this step can produce.
const s = resolve(`
INTENT-VERDICT: park
PARK-REASON: contradicted-by-code

## Spec

### Intent
Reported: the retry budget is shared between infra and semantic failures.

### Evidence
- retryAttempts and transientRetryAttempts are separate counters — src/work-driver-step-router.ts:192 — contradicted

## Rationale
The issue describes behaviour that was changed in #366. It is stale.
`);
assert(s?.parkReason === "contradicted-by-code", "a stale/contradicted issue parks as such");
assert(
s?.evidence.some((e) => e.verdict === "contradicted"),
"the contradicting evidence is retained for the handoff",
);
}

{
// The resolver is an LLM and can contradict itself. Evidence wins — ignoring
// a recorded contradiction is exactly how a wrong bug report gets built.
const s = resolve(`
INTENT-VERDICT: proceed

## Spec

### Intent
Fix the thing.

### Deliverables
- d1: change it [paths: src/a.ts]

### Evidence
- the described function does not exist — src/a.ts — contradicted
`);
assert(
s?.verdict === "park" && s.parkReason === "contradicted-by-code",
"a 'proceed' claimed alongside contradicting evidence is overridden to park",
);
}

// ---------------------------------------- already implemented / too large

{
const s = resolve(
"INTENT-VERDICT: park\nPARK-REASON: already-implemented\n\n## Spec\n\n### Intent\nAlready done.\n",
);
assert(s?.parkReason === "already-implemented", "already-implemented parks as such");
assert(/close/.test(parkAction("already-implemented", 9)), "its action is to confirm and close");
}
{
const s = resolve(
"INTENT-VERDICT: park\nPARK-REASON: too-large\n\n## Spec\n\n### Intent\nEverything.\n",
);
assert(s?.parkReason === "too-large", "too-large parks as such");
assert(/split/.test(parkAction("too-large", 9)), "its action is to split the issue");
}

// ------------------------------------------------- silence is not permission

{
// The inversion. Pre-#378, `work-driver-plan.ts` defaulted a missing verdict
// to NEEDS_WORK, so an agent that simply forgot the token got code written.
const s = resolve("## Spec\n\n### Intent\nSomething.\n\n### Deliverables\n- d1: do it\n");
assert(s?.verdict === "park", "a MISSING verdict parks — silence is not permission");
assert(s?.parkReason === "underspecified", "and carries an honest default reason");
}
{
const s = resolve("INTENT-VERDICT: banana\n\n## Spec\n\n### Intent\nx\n");
assert(s?.verdict === "park", "an unparseable verdict parks");
}
{
// A reply with no Spec block at all returns undefined, so runExplore falls
// back to the legacy router rather than parking every cycle on drift.
assert(
parseNormalisedSpec("VERDICT: NEEDS_WORK\n\nSome prose, no spec block.") === undefined,
"no `## Spec` block → undefined, so the legacy verdict router still applies",
);
}

// ------------------------------------------------------- assumptions

{
const s = resolve(`
INTENT-VERDICT: proceed-with-assumptions

## Spec

### Intent
Add a retry.

### Deliverables
- d1: add the retry [paths: src/a.ts]

### Assumptions
- Retry 3 times — matches the existing TRANSIENT_MAX_RETRIES elsewhere in the driver
`);
assert(s?.verdict === "proceed-with-assumptions", "gaps with defensible defaults proceed");
assert(
s?.assumptions[0]?.basis.includes("TRANSIENT_MAX_RETRIES") === true,
"the basis is parsed",
);
const block = renderAssumptions(s as NormalisedSpec);
assert(/## Assumptions made/.test(block), "assumptions render into the PR body");
assert(
/Retry 3 times/.test(block),
"the assumption text reaches review — otherwise 'proceed-with-assumptions' is not honest",
);
}
{
// Recording assumptions while claiming a plain `proceed` understates what
// review needs to see.
const s = resolve(
"INTENT-VERDICT: proceed\n\n## Spec\n\n### Intent\nx\n\n### Assumptions\n- assumed a default — no basis given\n",
);
assert(
s?.verdict === "proceed-with-assumptions",
"a plain 'proceed' carrying assumptions is promoted so they reach the PR body",
);
}
{
assert(
renderAssumptions({ assumptions: [] } as unknown as NormalisedSpec) === "",
"no assumptions → no block (no empty section in the PR body)",
);
}

// ------------------------------------------------------------ escape hatch

{
const prev = process.env.PI_ENSEMBLE_INTENT;
process.env.PI_ENSEMBLE_INTENT = "0";
try {
assert(!intentResolutionEnabled(), "PI_ENSEMBLE_INTENT=0 disables intent resolution");
} finally {
if (prev === undefined) delete process.env.PI_ENSEMBLE_INTENT;
else process.env.PI_ENSEMBLE_INTENT = prev;
}
assert(intentResolutionEnabled(), "and it is ON by default");
}

console.log(`\nexit ${exit}`);
process.exit(exit);
13 changes: 12 additions & 1 deletion extension/src/work-driver-commit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
mechanizeOpsEnabled,
withIntegrationLock,
} from "./work-driver-integrate.ts";
import { renderAssumptions } from "./work-driver-intent.ts";
import { parsePrNumber } from "./work-driver-lens.ts";
import { runSingleDispatch } from "./work-driver-merged.ts";
import { inlineCommitPrPrompt } from "./work-driver-prompts-late.ts";
Expand Down Expand Up @@ -142,13 +143,23 @@ export async function mechanizedCommitPr(
"every worktree was clean — no uncommitted work to consolidate (developer may not have written)",
};
}
// #378 — when the intent resolver filled gaps with defensible defaults,
// those assumptions belong where review happens. `proceed-with-assumptions`
// is only honest if the assumptions are visible; buried in a state file
// they may as well not exist.
const assumptionsBlock = ps.normalisedSpec
? renderAssumptions(ps.normalisedSpec as Parameters<typeof renderAssumptions>[0])
: "";
const prBody = [
"Automated by pi-ensemble /work driver (mechanized commit-pr).",
"",
...fixesLines,
...companionLines,
...workstreamLines,
].join("\n");
assumptionsBlock,
]
.filter((l) => l !== "")
.join("\n");
const prBodyFile = path.join(scratchDir(ctx.repoRoot, ctx.issue), "mech-pr-body.md");
await fs.mkdir(path.dirname(prBodyFile), { recursive: true });
await fs.writeFile(prBodyFile, prBody, "utf8");
Expand Down
13 changes: 13 additions & 0 deletions extension/src/work-driver-explain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
*/

import { MAX_CI_RETRIES, MAX_REVIEW_ROUNDS } from "./work-driver-context.ts";
import { type ParkReason, explainPark } from "./work-driver-intent.ts";
import type { WorkEvent, WorkState, WorkStep } from "./workflow-state.ts";

/**
Expand Down Expand Up @@ -41,6 +42,18 @@ export function explainCap(
return `developer subagent hit its wall-clock cap (PI_ENSEMBLE_SPAWN_TIMEOUT_MS_DEVELOPER, default 90 min) with ${fileBlurb} in the worktree — work needs different decomposition (split issue into smaller workstreams), a longer cap, or manual takeover`;
case "explore-already-complete":
return "explore concluded this issue is already done (e.g., satisfied by a prior PR or merged earlier). The driver halted before branch/develop ran — no code was written. Close the issue if you agree, or re-run /work with additional context if you believe there IS work to do";
case "intent-park": {
const spec = state.pipelineState.normalisedSpec;
const reason = (spec?.parkReason ?? "underspecified") as ParkReason;
const why = explainPark(reason, state.issue);
const contradictions = (spec?.evidence ?? []).filter((e) => e.verdict === "contradicted");
const evidence =
contradictions.length > 0
? `\n\nContradicting evidence:\n${contradictions.map((e) => ` - ${e.claim}${e.source ? ` (${e.source})` : ""}`).join("\n")}`
: "";
const rationale = spec?.rationale ? `\n\nResolver's rationale: ${spec.rationale}` : "";
return `${why} No code was written — the driver halted at intent resolution, before plan or branch ran.${evidence}${rationale}`;
}
case "existing-pr-detected": {
const pr = state.pipelineState.existingPr;
const via =
Expand Down
Loading