[finding] PINS_CHECKED detects only @ts-expect-error, so other compile-time pin idioms in a hidden test layer are invisible to it — measured on a live example #8113

Restart-when: any PR reworks the PINS_CHECKED detector half of scripts/check-type-check-coverage.mjs (the PIN_DIRECTIVE recogniser), or a second non-@ts-expect-error pin is measured proving phantom in a landed PR

Found while implementing #7809 (PR #8106). Out of scope there, filed unassigned.

What

scripts/check-type-check-coverage.mjs carries a PINS_CHECKED invariant with a good rationale:

A @ts-expect-error outside every invoked tsc program is a retirement guard that cannot fail — deleting the directive changes nothing, which is the definition of a phantom check.

It detects that condition by looking for @ts-expect-error directives (pinFiles). That is one idiom. A compile-time pin written any other way is not seen by it.

The measured example

While writing a vocabulary pin for #7809 I first expressed it as a mutual-assignability pair — the ordinary way to assert "these two types are the same" without a directive:

const _forward: Declared = undefined as unknown as EngineOp;   // red if EngineOp widens
const _reverse: EngineOp = undefined as unknown as Declared;   // red if EngineOp narrows

in packages/objectql/src/*.test.ts. packages/objectql/tsconfig.json excludes the test layer and its typecheck script is tsc --noEmit against that config, so the mandatory ablation (widen the union by one member, predict red) passed both pnpm typecheck and vitest. The pin asserted nothing in either runner a developer runs locally, and PINS_CHECKED had nothing to say about it because it carries no @ts-expect-error.

What this is NOT

Worth stating precisely, because the first version of this analysis was wrong and the correction matters:

These files are not unchecked. check:type-check-coverage's TEST_DEBT ledger lifts the exclusion, compiles the test layer, and ratchets the raw error count (@objectstack/objectql: 355). Verified from both sides on PR #8106 — three type errors in a new test file made it report 358 and go red; fixing them returned it to exactly 355. So a widened union would have been caught in CI.

The gap is therefore about signal, not coverage:

• what fires is TEST_DEBT +1 on a package with a 355-error pile, not "the dispatch vocabulary changed";
• it fires only in CI, not in the pnpm typecheck the author just ran locally and watched pass;
• PINS_CHECKED, the mechanism whose whole job is to say "this pin is a phantom", stays silent — so the author gets no warning that the idiom they chose is not load-bearing where they put it.

Why it is worth recording

The repo already decided this class matters (#5286, #5449, #4642, the PINS_CHECKED invariant, and scripts/check-test-typecheck.mts, which packages/spec and packages/client have onboarded to). This is a narrow hole in the detector, not a new class: an author who avoids @ts-expect-error — as anyone asserting type equality rather than expected-failure naturally does — falls outside it.

Non-obvious, and the reason this is filed rather than fixed in passing: the naive repair (flag any type-only const in a hidden test file) would be noisy and wrong. Something narrower is probably right — e.g. treat a package's hidden test layer as a place where compile-time pins are reported as unenforced-by-the-local-runner, or extend PINS_CHECKED's notion of a pin, or simply document at the ledger entry that pins in these packages should be expressed at runtime. Which of those is correct is a judgement call, hence a finding rather than a patch.

Ancillary datum

24 of 71 package tsconfigs exclude the test layer; a keyword scan finds type-level assertion idioms (@ts-expect-error, satisfies, assignability consts) in test files across ~12 of those packages. Most are presumably covered as counts by TEST_DEBT exactly as objectql is — that number is offered as scope context for whoever picks this up, not as a claim that any of them is broken.

Generated by Claude Code

Generated by Claude Code