Skip to content

Commit a888d1f

Browse files
committed
test(scripts): pin a battery roster floor on 18 class-1 self-tests
Batch 1b of #13799: the PR #13487 roster-floor shape transplanted onto the `scripts/**` self-tests whose assertion sink is NOT a block-bodied helper inside the self-test body -- a concise arrow, or a module-scope function -- so batch 1 (PR #14851) could not transplant it verbatim. Per the batch-1 review ruling, the roster machinery lives at MODULE scope (SELF_TEST_BATTERIES / SELF_TEST_BATTERY_FLOOR / UNATTRIBUTED_BATTERY / battery() / registerCase() / selfTestFloorFailures()), and each file's existing assertion sink is given a minimal block body that calls registerCase() and returns the original expression unchanged. No case is rewritten, none is reordered, and no assertion changes meaning: all 18 self-tests exit with the same code and byte-identical output before and after. What is pinned is the registered NAMES, not a total: every existing section banner opens a battery, every assertion is attributed to the battery most recently opened, the floor requires the OPENED set to equal the DECLARED set with each battery at or above its own count, and the roster's own size is pinned so deleting an entry cannot silence a floor quietly. One battery is pinned at its structural invariant rather than at today's count, with the reason written over the entry: check-plugin-teardown-shape's exclusions battery runs exactly one case per DELIBERATELY_EXCLUDED row, and promoting a name onto the teardown roster is a legitimate edit that shrinks that list. Census (`node scripts/measure-self-test-floor.mjs --json`): ROSTER 3 -> 21, NONE 158 -> 140; the set of files whose class changed equals this worklist exactly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015k1DVWthZyPS7xi1Q72YpK
1 parent 2263ca4 commit a888d1f

18 files changed

Lines changed: 2231 additions & 17 deletions

‎scripts/check-cli-command-ids.mjs‎

Lines changed: 119 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,109 @@ import { fileURLToPath } from 'node:url';
113113
import { execFileSync } from 'node:child_process';
114114
import { isEntrypoint } from './invoked-as.mjs';
115115

116+
// ── The self-test's own battery roster and floor (#13489) ──────────────────
117+
//
118+
// This self-test used to decide success by "no failure was recorded" and
119+
// nothing else, so "every case held" and "the cases never ran" printed the same
120+
// line. Closed the way PR #13487 validated on check-doc-authoring: what is
121+
// pinned is the registered NAMES, not a number. Every section opens with
122+
// `battery('<name>')`, every assertion is attributed to the battery most
123+
// recently opened, and the floor requires the OPENED set to equal the DECLARED
124+
// set with each battery at or above its own count.
125+
//
126+
// ⛔ A pinned TOTAL is not the repair: a battery dropping from 9 cases to 3 keeps
127+
// a total "right" the moment a sibling grows. A set difference says WHICH
128+
// battery stopped; a count says only that something did.
129+
//
130+
// The counts are a FLOOR, not an equality — adding cases is ordinary work and
131+
// must not red. A battery BELOW its floor means cases stopped running; the
132+
// remedy is to find what stopped registering.
133+
//
134+
// The machinery lives HERE, at module scope, rather than inside the self-test:
135+
// this self-test's assertion sink is not a block-bodied helper in its body (it
136+
// is a concise arrow, or a module-scope function), so there is no in-body
137+
// helper to thread a per-run ledger through. Module scope is safe because the
138+
// self-test runs once per process, and it is what lets the existing sink route
139+
// through `registerCase()` with no case rewritten and no assertion changed.
140+
const SELF_TEST_BATTERIES = Object.freeze({
141+
'the id derivation, against a scratch tree (no repo state)': 12,
142+
'a KNOWN-BAD literal: a command id that resolves to nothing': 3,
143+
'the delimiter rule: the six measured noise shapes stay OUT': 6,
144+
'the exemption ledger is site-scoped, not blanket': 3,
145+
'the ledger self-retires: a listed entry that stops reproducing REDS': 3,
146+
'the dispatch-gates declaration (#12016\'s own landing obligation)': 5,
147+
'bin names come from declared data': 3,
148+
'the live repo returns a verdict, and it is green': 4,
149+
});
150+
151+
// DELETING an entry silences that battery's floor exactly as effectively as
152+
// zeroing it, so the roster's own size is pinned too.
153+
const SELF_TEST_BATTERY_FLOOR = 8;
154+
155+
// The key an assertion is filed under when no battery is open. It is not a
156+
// declared battery, so it reds by the same set difference rather than silently
157+
// inflating whichever battery happened to run last.
158+
const UNATTRIBUTED_BATTERY = '(no battery open)';
159+
160+
/** Cases registered per battery: `battery()` opens one, `registerCase()` files into it. */
161+
const selfTestSeen = new Map();
162+
let selfTestOpenBattery = null;
163+
164+
/** Open a battery. Every assertion after this line is attributed to it. */
165+
function battery(name) {
166+
selfTestOpenBattery = name;
167+
}
168+
169+
/** Called by the self-test's own assertion sink, once per assertion. */
170+
function registerCase() {
171+
const name = selfTestOpenBattery ?? UNATTRIBUTED_BATTERY;
172+
selfTestSeen.set(name, (selfTestSeen.get(name) ?? 0) + 1);
173+
}
174+
175+
/**
176+
* The floor: every declared battery RAN, and ran its cases (#13489).
177+
*
178+
* Evaluated after every battery has had its chance and BEFORE the verdict, so
179+
* the success line can only be printed by a run in which the set of batteries
180+
* that registered assertions EQUALS the set declared.
181+
*/
182+
function selfTestFloorFailures() {
183+
const declared = Object.keys(SELF_TEST_BATTERIES);
184+
const problems = [];
185+
if (declared.length < SELF_TEST_BATTERY_FLOOR) {
186+
problems.push(
187+
`SELF_TEST_BATTERIES declares ${declared.length} batteries, below the pinned `
188+
+ `${SELF_TEST_BATTERY_FLOOR} — a battery deleted from the roster takes its own floor with it.`,
189+
);
190+
}
191+
for (const [name, count] of selfTestSeen) {
192+
if (declared.includes(name)) continue;
193+
problems.push(
194+
`self-test battery "${name}" registered ${count} case(s) but is not declared in `
195+
+ 'SELF_TEST_BATTERIES — an assertion attributed to no declared battery is one nothing floors.',
196+
);
197+
}
198+
for (const name of declared) {
199+
const count = selfTestSeen.get(name) ?? 0;
200+
if (count >= SELF_TEST_BATTERIES[name]) continue;
201+
problems.push(
202+
count === 0
203+
? `self-test battery "${name}" DID NOT RUN — 0 cases registered, ${SELF_TEST_BATTERIES[name]} pinned. `
204+
+ 'The verdict below would have claimed those cases hold.'
205+
: `self-test battery "${name}" registered ${count} case(s), below its pinned floor of `
206+
+ `${SELF_TEST_BATTERIES[name]} — cases that used to run no longer do.`,
207+
);
208+
}
209+
if (problems.length) {
210+
problems.push(
211+
'A battery at or below its floor means cases STOPPED RUNNING — the battery is the bug, not the '
212+
+ 'number. Find what stopped registering (an early return, a deleted block, a guard that now '
213+
+ 'skips) and restore it.',
214+
);
215+
}
216+
return problems;
217+
}
218+
116219
const REPO_ROOT = join(dirname(fileURLToPath(import.meta.url)), '..');
117220
const OCLIF_COMMANDS_DIR = 'src/commands';
118221

@@ -453,9 +556,13 @@ let selfTestReachedVerdict = false;
453556

454557
function selfTest() {
455558
const cases = [];
456-
const t = (name, ok, detail = '') => cases.push({ name, ok, detail });
559+
const t = (name, ok, detail = '') => {
560+
registerCase();
561+
return cases.push({ name, ok, detail });
562+
};
457563

458564
// -- the id derivation, against a scratch tree (no repo state) --------------
565+
battery('the id derivation, against a scratch tree (no repo state)');
459566
const dir = mkdtempSync(join(tmpdir(), 'cli-cmd-ids-'));
460567
try {
461568
const cmds = join(dir, 'src', 'commands');
@@ -499,6 +606,7 @@ function selfTest() {
499606
}
500607

501608
// -- a KNOWN-BAD literal: a command id that resolves to nothing -------------
609+
battery('a KNOWN-BAD literal: a command id that resolves to nothing');
502610
const ids = new Set(['migrate apply', 'migrate', 'build']);
503611
const topics = new Set(['migrate']);
504612
const bad = literalsOn("throw new Error('run \"os migrate nonexistent-command\" first');", ['os']);
@@ -508,6 +616,7 @@ function selfTest() {
508616
resolveId(literalsOn('`os migrate apply`', ['os'])[0].words, ids, topics) === 'migrate apply');
509617

510618
// -- the delimiter rule: the six measured noise shapes stay OUT ------------
619+
battery('the delimiter rule: the six measured noise shapes stay OUT');
511620
t('Spanish prose ("envios diarios") is not a literal', literalsOn("label: 'Limite de envios diarios',", ['os']).length === 0);
512621
t('a Python import example is not a literal', literalsOn("import os from 'os';", ['os']).length === 0);
513622
t('an unquoted sentence is not a literal', literalsOn('`carry an os validate-clean security posture`,', ['os']).length === 0);
@@ -516,11 +625,13 @@ function selfTest() {
516625
t('a bin name at a quote IS a literal', literalsOn('via "os migrate apply --allow-destructive".', ['os']).length === 1);
517626

518627
// -- the exemption ledger is site-scoped, not blanket ----------------------
628+
battery('the exemption ledger is site-scoped, not blanket');
519629
t('a declared fixture is exempt', isExempt('scripts/docs-audit/check-drift-comment.mjs', 'os demo'));
520630
t('the SAME text elsewhere is NOT exempt', !isExempt('packages/drivers/driver-sql/src/schema-drift.ts', 'os demo'));
521631
t('a DIFFERENT text in an exempt file is NOT exempt', !isExempt('scripts/docs-audit/check-drift-comment.mjs', 'os migrate gone'));
522632

523633
// -- the ledger self-retires: a listed entry that stops reproducing REDS ---
634+
battery('the ledger self-retires: a listed entry that stops reproducing REDS');
524635
t('every ledger entry reproduces in the live scan', audit().stale.length === 0,
525636
audit().stale.map((e) => `${e.file} "${e.text}"`).join('; '));
526637
t('a fabricated ledger entry would be reported stale',
@@ -546,6 +657,7 @@ function selfTest() {
546657
// forever and pays itself out as a dev dispatched on a scripts/ card with this gate
547658
// missing from the brief. Both directions are pinned, and both matter — a missing
548659
// declaration is a silent gate, a surplus one is a lying gate.
660+
battery('the dispatch-gates declaration (#12016\'s own landing obligation)');
549661
const separatorless = POPULATION_ROOTS.filter((r) => !r.includes('/'));
550662
t('every whole-root population entry is declared as a subtree (a bare root is refused by '
551663
+ 'hintCovers as too generic, so it needs the `<root>/**` spelling)',
@@ -588,11 +700,13 @@ function selfTest() {
588700
&& new Set(audit().resolved.filter((x) => x.file.startsWith('scripts/')).map((x) => x.file)).size >= 5);
589701

590702
// -- bin names come from declared data ------------------------------------
703+
battery('bin names come from declared data');
591704
t('oclif.bin is read', binNamesOf({ oclif: { bin: 'os' } }).includes('os'));
592705
t('bin keys join it', binNamesOf({ oclif: { bin: 'os' }, bin: { objectstack: './bin/run.js' } }).includes('objectstack'));
593706
t('a package with no oclif block declares no bins', binNamesOf({ bin: { foo: 'x' } }).length === 0);
594707

595708
// -- the live repo returns a verdict, and it is green ----------------------
709+
battery('the live repo returns a verdict, and it is green');
596710
const live = audit();
597711
t('the live audit returns a verdict', live.refusal === null, live.refusal ?? '');
598712
t('the live repo has at least one CLI package', live.refusal === null && live.clis.length >= 1);
@@ -602,6 +716,10 @@ function selfTest() {
602716
live.refusal === null && live.resolved.some((x) =>
603717
x.file === 'packages/drivers/driver-sql/src/schema-drift.ts' && x.text === 'os migrate multi-value-columns'));
604718

719+
// The floor runs BEFORE the verdict below, so a success line can only be
720+
// printed by a run in which every declared battery registered its cases.
721+
for (const message of selfTestFloorFailures()) cases.push({ name: message, ok: false, detail: '' });
722+
605723
const failed = cases.filter((c) => !c.ok);
606724
for (const c of failed) console.error(` ✗ ${c.name}${c.detail ? ` -- ${c.detail}` : ''}`);
607725
if (failed.length) {

0 commit comments

Comments
 (0)