Skip to content

Verify-full tier (.pi/verify-cmd-full at ci step) + deterministic type-widening scan routed to lens context #279

Description

@randomm

Context

Verified findings (2026-07-26), two independent gaps that ship well together (zero file overlap with the develop-gates issue):

  1. No fast/full verify tier. verifyCmdFor (work-driver.ts ~:2257-2305) resolves one command (.pi/verify-cmd > package.json typecheck > Cargo.toml > test). The harness cannot distinguish "the fast suite passed" from "the suite that exercises real dependencies passed." In vipune, the fast suite passed for ~2.5 months while the real-embedder tests sat behind #[ignore] — "fast green, full unrun" was invisible.

  2. Constraint removal is not an event. The vipune bug's mechanism: EmbeddingEngineOption<EmbeddingEngine> (commit ea8c836) removed a compiler-enforced invariant; a later change (fa9430c) read the Option as an invitation to write a mock-fallback None branch in production. Nothing in the pipeline treats type-widening as signal. No diff-level deterministic scanner exists in pi-ensemble today (verified).

This work must ship as its own separate PR, independent of any other open issue.

What to build

A. Verify-full tier.

  • verifyCmdFullFor(repoRoot): reads .pi/verify-cmd-full ONLY — first non-empty non-comment line, verbatim. NO derivation fallback (an inferred "full suite" recreates exactly the ambiguity this removes). Absent file → tier skipped with a visible note.
  • Execute driver-side in runCi BEFORE the ops gh run watch dispatch, via the existing verifyExecFn ?? execp seam, with its own timeout env PI_ENSEMBLE_VERIFY_FULL_TIMEOUT_MS (default 30 min).
  • New schema-additive event verify-full-status { status: "success" | "failure" | "skipped", at, ms?, evidenceTail? } in workflow-state.ts. On failure: bump ciRetryCount and skip the ops dispatch for this round (the failure routes through the existing ci-retry machinery); the PR comment / handoff must show fast and full results SEPARATELY so "fast green, full unrun/red" is visible.

B. Type-widening scan.

  • New pure module extension/src/invariant-scan.ts exporting scanTypeWidening(diff: string): Array<{file, line, kind, before?, after?}>. Deterministic regex patterns over added/removed diff lines:
    • TOption<T> (Rust), TT | null / T | undefined (TS), property gaining ?:
    • removed readonly / final / const / NOT NULL
    • type narrowed to any / unknown / interface{}
    • removed assert / debug_assert! / invariant-check lines
  • Wire in runLens between fetchAllMergedDiffs (:2564) and the lens call (:2606): findings appended to the lens context as mandatory input with the framing "the ARCHITECTURE lens must answer: what invariant did this widening remove, and what now guarantees it?" Route, never block — the scan attaches context; it does not fail the cycle. Also emit an additive widening-scan event with the findings for audit.

Env: PI_ENSEMBLE_VERIFY_FULL=0, PI_ENSEMBLE_WIDENING_SCAN=0 (default ON).

Acceptance criteria

  • scanTypeWidening fires on a fixture diff containing the exact vipune ea8c836 shape (embedder: EmbeddingEngineembedder: Option<EmbeddingEngine>), and on TS | null widening, added ?:, removed readonly, and removed debug_assert! fixtures; does NOT fire on unrelated diffs (precision fixtures).
  • Scanner findings provably reach the lens context (injected lensReviewFn captures the context; assert the framing line present).
  • With .pi/verify-cmd-full present and failing, runCi emits verify-full-status: failure, bumps ciRetryCount, and does NOT dispatch ops that round — end-to-end via injected verifyExecFn/dispatchFn.
  • With the file absent, verify-full-status: skipped is emitted (visible, not silent).
  • Green full suite → verify-full-status: success then the normal ops CI watch proceeds.
  • Both env kill-switches restore today's behaviour exactly.
  • tsc + biome + all 26 offline smokes green; v1 state files load unchanged (events are additive).
  • Docs: troubleshooting section for both features; AGENTS.md §7 sentence.

Out of scope

  • Blocking on widening findings (route-only until lens precision is measured — see calibration issue).
  • Writing vipune memories on widening events (separate follow-up issue reuses this scanner).
  • Any derivation fallback for the full command.

Test plan

Offline: scanner fixture suite in test-work-driver.ts (or a small dedicated test file wired into the AGENTS.md §1 loop); runCi integration tests with injected seams. Live (nessie): add .pi/verify-cmd-full (e.g. cargo test --workspace), run /work N, confirm the PR comment shows fast + full separately; craft a widening diff and confirm the ARCHITECTURE lens received the mandate.


Addendum (2026-07-27) — cwd hazard under the parallel-work series (#287#290)

The verify-full tier (part A) executes the full suite driver-side in runCi. Under parallel groups (#289), repoRoot may be checked out on a DIFFERENT group's feature branch when this group's ci step runs — repoRoot HEAD is only guaranteed to be this group's branch inside the integration mutex, which has long since released by ci time. Executing .pi/verify-cmd-full at repoRoot could therefore test the wrong branch.

Requirement: run the verify-full command in the group's primary worktree (worktrees[first id], which after #287 is always a real worktree on the group's branch state), never at repoRoot. Acceptance criteria addition:

  • verify-full executes with cwd = the group's worktree; an injected-exec test asserts no verify-full invocation has cwd === repoRoot.

Addendum (2026-08-05) — interrupted cycle, WIP base, and a hollow-test defect to fix on resume

A /work 279 cycle was interrupted mid-develop by a provider 429 (per-minute rate limit). Partial work was type-checked, linted, and preserved as WIP commit 021651b on branch feature/issue-279-verify-tier-and-type-scan, cut from fc061db. The issue remains open (Refs #279, not Fixes). That commit is not reviewed — it has passed neither the adversarial gate nor the six-pass lens review.

Landed in 021651b:

  • extension/src/invariant-scan.ts (234 lines) — scanner module
  • extension/src/work-driver-verify-full.ts (72 lines) — verify-full tier
  • extension/smoke-tests/test-invariant-scan.ts (400 lines)
  • extension/smoke-tests/test-verify-full.ts (202 lines)
  • modified: work-driver-lens.ts, work-driver-stepback-ci.ts, work-status.ts, workflow-state-events.ts (adds the additive verify-full-status and widening-scan event variants)

BLOCKING defect in the WIP — both new smoke tests are hollow

test-invariant-scan.ts:375 and test-verify-full.ts:186 each declare export async function run() { ... } and never invoke it. Executing either file defines its test functions, runs none, prints nothing, and exits 0. They were counted as passing on the strength of producing no output.

The assertions themselves are real (89 and 33 assertion sites; 23 test cases in the scanner suite) — only the top-level invocation is missing. The project convention is top-level execution ending in process.exit(exit); see extension/smoke-tests/test-work-status.ts.

This is EPIC #328's thesis reproduced inside the issue meant to strengthen verification tiers: a gate that cannot fail is not a gate.

Required on resume:

  • Both new smoke tests execute at module top level and exit non-zero on assertion failure, following the test-work-status.ts convention.
  • Each new test is proven failable — mutate the module under test, observe a non-zero exit, revert.
  • The acceptance criterion above reading "all 26 offline smokes green" is stale; the suite is now 57 files. Treat the count as "every offline test-*.ts excluding *-live.ts".

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions