Skip to content

fix(work-driver): 23 sites pattern-match agent output for meaning, and several fail confidently in the wrong direction #408

Description

@randomm

Context

An audit of extension/src/ for "regex applied to an LLM reply or a doctrine file, to extract a decision" found 24 sites. Three are already fixed or in flight (#397's parseEvidence, #404's PARK-REASON, #407's GRANT_PATTERNS). This issue covers the rest, ranked by whether a no-match yields a confident wrong answer rather than an obvious failure.

The shared property that makes these dangerous: the failure is silent, and the substituted default is then acted on as though it were read.

Tier 1 — a no-match produces a wrong confident answer

parsePerIssueVerdicts defaults to build-itwork-driver-plan.ts:275. On no match for an issue, falls back to the overall verdict; if that is absent too, NEEDS_WORK. This is precisely the pre-#378 "silence is permission" that work-driver-intent.ts:13-14 claims was structurally removed — it survives untouched on the multi-issue path, which #397 made the only multi-issue path. The reason string at :277-278 is honest ("(no verdict; defaulting to NEEDS_WORK)") and nothing gates on it.

labelApplied fails openwork-driver-handoff.ts:116, /label.*needs-human-attention/i over ops prose. An ops agent writing "I could not apply the label needs-human-attention" matches, so the gh fallback is skipped and the driver records success on an issue that was never labelled.

parseVerdict hands over the wrong textadversarial.ts:376-380. Case-sensitive, no bold tolerance (unlike both siblings at work-driver-plan.ts:235 and work-driver-intent.ts:128), so Verdict: Approved misses. On a miss the default is ISSUES_FOUND and findings is set to the entire reply (:379) — so the fix-developer is handed the whole approval message as "findings to fix", and burns a round on it.

ci-status has no tolerance at allwork-driver-stepback-ci.ts:260-268, a bare text.includes(). ci-status:success, CI-status: success, or the marker inside a table all miss. No-match → "failure" → bumps ciRetryCount and can fire the ci-retry cap. A green CI reported in a slightly different shape burns the retry budget and parks the cycle. The #380 demotion check at :274-286 only runs when the status is already success, so it cannot rescue a false failure.

parseAdversarialRounds invents a numberwork-driver-adversarial.ts:65-69. Three includes() on prose, defaulting to 3, asserted into the event log as fact with no "unknown" state. A phrasing change in adversarial.ts's own synthesised text silently makes every run report three rounds.

Grouping R1/R3/R4work-driver-grouping.ts:122, :171, :243. Two of these were structurally impossible to satisfy and were only found by accident (#376). Live evidence from nessie: #657 said "Closes the drift in #650" and R1's alternation has no closes/fixes/resolves, so two issues where one fixes the other ran as concurrent groups. And .container is absent from R2's extension list (:188), so selfhost/nessie.container — the one file both issues genuinely shared — contributed nothing to the Jaccard.

sliceSubsection / sliceMarkdownSectionwork-driver-intent.ts:96, work-driver-plan.ts:398. Require an exact ^###\s+<name>\s*$. A resolver writing ### Deliverables:, #### Deliverables or **Deliverables** yields undefined[]specIsComplete false → park. Silent; nothing traces that a heading was not found.

Tier 2 — degrades quietly, failure at least structural

parseWorkstreams (work-driver-plan.ts:301) — a miss synthesises default with paths: [], which disables verifyConsolidation's oracle entirely (work-driver-verify.ts:70-73). A parse miss silently turns off the diff-coverage check.

parseAbort (work-driver-diff.ts:281) — only scans the last 800 chars, so an abort stated early in a long reply is invisible, and the step is recorded ok: true — the literal #553 incident its own docstring describes.

lens-review-format.ts:72-82 — an unrecognised severity string drops the finding entirely, counted only as an integer. A CRITICAL with severity: "BLOCKER" vanishes and the verdict computes as if it never existed.

Adjacent: the driver re-parsing its own enums

work-queue.ts:144-181 regexes cap-reason strings the driver itself produced — structured enums round-tripped through a string and re-parsed. A cap rename falls through to the generic --restart advice at :181, which for a merge hold is actively destructive (documented at :130-134).

The good pattern, already in the codebase

Two sites show the way out:

  • lens-review-format.ts:40-94 asks for a tool call with a typed schema Pi validates in the child: "No text parsing involved… malformed calls never reach this code."
  • parsePrNumber (work-driver-lens.ts:31) has a real executed-evidence fallback — on a miss, gh pr list --head <branch> recovers the number. It is the only parser in the codebase that repairs itself from reality rather than from a default.

Acceptance criteria

  • Each site fixed carries a canary that fails against main.
  • parsePerIssueVerdicts no longer defaults to build-it — silence parks, matching feat(work-driver): resolve intent from any spec — ground it against code and world, then proceed, assume, or park #378's stated principle.
  • labelApplied is established by executed evidence (gh issue view --json labels), not by prose.
  • parseVerdict tolerates case and bold; a miss does not pass the whole reply as findings.
  • ci-status tolerance matches its siblings; a formatting miss cannot park a green cycle.
  • R1 accepts closes/fixes/resolves/fixed by/superseded by; R2's extension list covers .container/.service/.unit/Dockerfile.
  • work-queue.ts routes on the structured cap value, not a re-parsed string.
  • Quality gate per AGENTS.md §1; docs updated.

Sequencing

Independent of #406 and #407 and can proceed in parallel, but lower priority than both — none of these silently escalates authority. Best split across several PRs by tier rather than landed as one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions