Skip to content

Commit 20d61f7

Browse files
committed
feat(qa-checklist): resolve provisioning.use against its own area's recipes
`docs/qa/platform-checklist/README.md` recorded option C from #7716 as deliberately deferred (tracked at #7720, which landed only the documentation half), "to be revisited if the recipe shape spreads to more areas". It has: recipes now live in three area files and six items carry a `provisioning.use`. Until now a `use` naming a key its area does not define validated clean — measured on 112a8c6, a typo'd `qa-media-constraint` and a cross-area `qa-contributor-bound-member` each exited 0 with the untouched OK line. An item that reads as provisioned and is not costs the run the clauses the recipe was meant to unblock, mid-run and on a live boot. The resolve mirrors the existing `supersededBy` check — same `err(file, id, …)` reporting, same "points at unknown" wording — and reuses the trap checker's did-you-mean, since a typo of a real recipe is the drift shape review is worst at. It carries its own inline positive-control battery (14 assertions) for the reason the trap battery does: this gate is not CI-wired, so a check that quietly stopped firing would restore exactly the green it replaced. Scope is gap 1 of #10593 only. Cross-area reuse still has no spelling, so the resolve is same-area only and the unreferenced-recipe direction is deliberately NOT checked — redding it would settle that convention question by accident. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
1 parent 112a8c6 commit 20d61f7

2 files changed

Lines changed: 173 additions & 10 deletions

File tree

‎docs/qa/platform-checklist/README.md‎

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -165,10 +165,30 @@ Why this shape:
165165
*CLOSED-by-recipe*, naming any pinned fallback and asking the run to record **which**
166166
of the two its verdict rests on. Deleting the gap loses the reason the recipe exists.
167167

168-
The validator does **not** yet resolve `provisioning.use` against the area's `fixtures`
169-
keys — that was deliberately deferred (option C on #7716's open question, tracked at
170-
#7720), to be revisited if the recipe shape spreads to more areas. Until then a typo'd
171-
`use` is caught by review, not by `check:platform-checklist`: copy the key, don't retype it.
168+
The validator **resolves `provisioning.use` against its own area's `fixtures` keys**: a
169+
`use` naming a key that area does not define fails `check:platform-checklist`, naming the
170+
item, the key that resolved to nothing, and the recipes the area does offer. This is
171+
option C on #7716's open question — deferred at #7720 while the recipe shape lived in a
172+
single area, landed at #10593 on its own stated condition, once the shape had spread to
173+
three areas and six references.
174+
175+
Two things the resolve deliberately does **not** do:
176+
177+
- **It does not flag a recipe no item references.** Cross-area reuse has no spelling yet
178+
(below), so a recipe whose only consumer lives in another area is referenced from that
179+
item's `knownGaps` prose — invisible to the check. Redding the unreferenced direction
180+
would answer the cross-area question by accident, in the direction of "recipes are
181+
area-local", and that is a convention decision rather than a mechanical one.
182+
- **It does not reach across areas.** Resolution is area-scoped because the mechanism is:
183+
`use` names a key in the item's *own* file. A `use` pointing at another area's recipe
184+
key is therefore a dangling pointer and fails — there is no qualified spelling
185+
(`search:qa-contributor-bound-member` or similar) and no shared recipe file. A
186+
cross-area consumer still cites the recipe **by name in `knownGaps`** and does not fork
187+
a second copy; `records-forms.crud-roundtrip` clause 7 is the worked instance. Giving
188+
that pointer a spelling the tooling can see is the open half of #10593.
189+
190+
⚠️ Remember the cadence: this gate is **not** CI-wired (above), so it catches a typo'd
191+
`use` at the next manual run, not on the PR that introduced it. Copy the key, don't retype it.
172192

173193
## Lifecycle — append, change, retire (never delete)
174194

‎scripts/check-platform-checklist.mjs‎

Lines changed: 149 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,10 @@
3737
// vocabulary is READ from that table, never copied into this file, and a
3838
// table this script cannot parse is a refusal rather than an empty
3939
// allow-list (see the trap-vocabulary block below for why that matters).
40+
// - every `fixtures.provisioning.use` resolves to a recipe key in its OWN
41+
// area's area-level `fixtures` block — an item that reads as provisioned
42+
// and is not costs the run the clauses the recipe was meant to unblock,
43+
// mid-run and on a live boot (see the provisioning block below).
4044
//
4145
// It does NOT judge whether an item is testable or its oracle sufficient — no
4246
// static check can. It guarantees the *structure* a run can be trusted against.
@@ -220,6 +224,71 @@ function trapProblems(item, vocabulary) {
220224
return out;
221225
}
222226

227+
// ── Area-scoped provisioning recipes (`fixtures.provisioning.use`) ──────────
228+
// An area may write ONE provisioning recipe at the area level and have many
229+
// items opt into it by key (README "Area-level `fixtures` — one named
230+
// provisioning recipe, many items"). Both halves have to be real: a recipe
231+
// nobody references is dead text, and a `use` naming a key that is not there
232+
// is a dangling pointer — an item that READS as provisioned and is not. The
233+
// run finds out at the worst possible moment: mid-run, on a live boot, with
234+
// the clauses the recipe was supposed to unblock now scoring blocked(fixture).
235+
//
236+
// This resolves the dangling direction. It was deliberately deferred while the
237+
// recipe shape lived in exactly one area (option C on #7716's open question,
238+
// tracked at #7720), on the stated condition that it be revisited if the shape
239+
// spread. It has: three area files, six references (#10593).
240+
//
241+
// Scope, deliberately narrow in two directions:
242+
//
243+
// 1. SAME-AREA resolution only, because that is the mechanism the README
244+
// documents. Cross-area reuse has no spelling at all today (the second,
245+
// undecided gap on #10593), so a `use` pointing at another area's recipe
246+
// key IS a dangling pointer here — and the message says so, instead of
247+
// letting a reference that resolves nowhere read as if it worked.
248+
// 2. The reverse direction — a recipe no item references — is NOT checked,
249+
// unlike the bidirectional trap vocabulary above. With cross-area reuse
250+
// unspelled, a recipe whose only consumer sits in another area is cited
251+
// from that item's `knownGaps` prose; redding it here would settle gap 2
252+
// by accident, in the direction of "recipes are area-local", which is a
253+
// convention decision and not this check's to make.
254+
//
255+
// `$`-prefixed keys are annotations, not recipes — every area `fixtures` block
256+
// opens with a `$comment` stating the block's purpose and replay rule — so
257+
// they are excluded from the recipe set, and from the did-you-mean.
258+
259+
/** The recipe keys of one area doc's area-level `fixtures` block. `$…` keys are annotations, not recipes. */
260+
function areaRecipeKeys(doc) {
261+
const f = doc?.fixtures;
262+
if (!f || typeof f !== 'object' || Array.isArray(f)) return [];
263+
return Object.keys(f).filter((k) => !k.startsWith('$'));
264+
}
265+
266+
/** Problems with one item's `fixtures.provisioning`, as message strings. Pure; battery-tested below. */
267+
function provisioningProblems(item, recipeKeys) {
268+
const out = [];
269+
const p = item?.fixtures?.provisioning;
270+
if (p === undefined) return out; // optional field: 6 of 205 items opt into a recipe
271+
if (typeof p !== 'object' || p === null || Array.isArray(p) || typeof p.use !== 'string' || !p.use.trim()) {
272+
out.push('"fixtures.provisioning" must carry a non-empty string "use" naming a recipe in this area\'s area-level "fixtures" block — a provisioning block that opts into nothing reads as provisioned and is not');
273+
return out;
274+
}
275+
const recipes = new Set(recipeKeys);
276+
if (recipes.has(p.use)) return out;
277+
if (recipes.size === 0) {
278+
out.push(
279+
`"fixtures.provisioning.use" names "${p.use}" but this area file has no area-level "fixtures" block to resolve it against` +
280+
' — write the recipe as a sibling of "area"/"title"/"items" (README "Area-level `fixtures`"), or drop the reference.',
281+
);
282+
return out;
283+
}
284+
out.push(
285+
`"fixtures.provisioning.use" names "${p.use}", which is not a recipe in this area's area-level "fixtures" block${didYouMean(p.use, recipes)}` +
286+
` — this area offers ${[...recipes].map((k) => `\`${k}\``).join(', ')}. References are AREA-SCOPED: a recipe another area owns cannot be opted into by key` +
287+
' (cross-area reuse has no spelling yet, #10593) — cite that recipe by name in this item\'s "knownGaps" instead, and do not fork a second copy.',
288+
);
289+
return out;
290+
}
291+
223292
/**
224293
* The positive control. Proves the extractor reads a good table AND refuses an
225294
* empty / renamed / reshaped one, and that the item-side checker catches both
@@ -289,14 +358,64 @@ function selfTestTrapVocabulary() {
289358
return { checked, failures };
290359
}
291360

361+
/**
362+
* The positive control for the provisioning resolve — same shape and the same
363+
* reason as the trap battery above. This check's entire value is that it
364+
* FIRES, and the failure it prevents is invisible from the outside: measured
365+
* on `main` at 112a8c6731, a typo'd `use` (`qa-media-constraint` for
366+
* `qa-media-constraints`) and a cross-area `use` each validated clean, exit 0,
367+
* printing the same OK line as an untouched tree. A check that quietly stopped
368+
* firing would restore exactly that green. Zero I/O — every subject literal.
369+
*/
370+
function selfTestProvisioningUse() {
371+
const failures = [];
372+
let checked = 0;
373+
const t = (what, ok) => {
374+
checked++;
375+
if (!ok) failures.push(what);
376+
};
377+
378+
const keys = areaRecipeKeys({ fixtures: { $comment: 'what this block is, and the replay rule', 'qa-scratch-authz': {}, 'qa-media-constraints': {} } });
379+
const item = (use) => ({ fixtures: { app: 'showcase', provisioning: { use, why: 'which clauses it unblocks' } } });
380+
381+
t('U1 a `use` naming a recipe of this area passes', provisioningProblems(item('qa-scratch-authz'), keys).length === 0);
382+
t('U2 an item whose fixtures carry no provisioning is fine (optional field)', provisioningProblems({ fixtures: { app: 'showcase' } }, keys).length === 0);
383+
t('U3 an item with no fixtures block at all is fine', provisioningProblems({}, keys).length === 0);
384+
385+
const dangling = provisioningProblems(item('qa-recipe-nobody-wrote'), keys);
386+
t('U4 a `use` no recipe answers to is flagged', dangling.length === 1 && dangling[0].includes('qa-recipe-nobody-wrote'));
387+
388+
const typo = provisioningProblems(item('qa-media-constraint'), keys);
389+
t('U5 a TYPO of a real recipe is flagged — the drift shape review is worst at', typo.length === 1);
390+
t('U6 the typo message names the recipe that was meant', typo.length === 1 && typo[0].includes('did you mean `qa-media-constraints`'));
391+
t('U7 the message lists the recipes this area does offer', typo.length === 1 && typo[0].includes('`qa-scratch-authz`'));
392+
393+
t('U8 a recipe key ANOTHER area owns does not resolve here (references are area-scoped)', provisioningProblems(item('qa-contributor-bound-member'), keys).length === 1);
394+
t('U9 `$comment` is an annotation, not a recipe', !keys.includes('$comment') && provisioningProblems(item('$comment'), keys).length === 1);
395+
t('U10 a whitespace-padded spelling does not resolve', provisioningProblems(item('qa-scratch-authz '), keys).length === 1);
396+
t('U11 a provisioning block with no "use" is flagged', provisioningProblems({ fixtures: { provisioning: { why: 'because' } } }, keys).length === 1);
397+
t('U12 a non-string "use" is flagged', provisioningProblems({ fixtures: { provisioning: { use: 42 } } }, keys).length === 1);
398+
399+
const noBlock = provisioningProblems(item('qa-scratch-authz'), areaRecipeKeys({ area: 'records-forms', items: [] }));
400+
t('U13 an area with NO fixtures block says so, rather than offering an empty list', noBlock.length === 1 && noBlock[0].includes('no area-level "fixtures" block'));
401+
t('U14 a fixtures block holding only annotations exposes zero recipes', areaRecipeKeys({ fixtures: { $comment: 'x' } }).length === 0);
402+
403+
return { checked, failures };
404+
}
405+
292406
if (process.argv.slice(2).includes('--self-test')) {
293-
const r = selfTestTrapVocabulary();
294-
if (r.failures.length === 0) {
295-
console.log(`✓ check-platform-checklist --self-test: ${r.checked} assertions — the trap-table extractor reads a good table and REFUSES an empty/renamed/reshaped one.`);
407+
const trap = selfTestTrapVocabulary();
408+
const prov = selfTestProvisioningUse();
409+
const failures = [...trap.failures, ...prov.failures];
410+
if (failures.length === 0) {
411+
console.log(
412+
`✓ check-platform-checklist --self-test: ${trap.checked + prov.checked} assertions — the trap-table extractor reads a good table and REFUSES an empty/renamed/reshaped one;` +
413+
' `fixtures.provisioning.use` resolves against its own area and fires on a dangling one.',
414+
);
296415
process.exit(0);
297416
}
298-
console.error(`✗ check-platform-checklist --self-test — ${r.failures.length} failure(s)\n`);
299-
for (const f of r.failures) console.error(` • ${f}`);
417+
console.error(`✗ check-platform-checklist --self-test — ${failures.length} failure(s)\n`);
418+
for (const f of failures) console.error(` • ${f}`);
300419
process.exit(1);
301420
}
302421

@@ -308,6 +427,15 @@ if (trapControl.failures.length) {
308427
process.exit(1);
309428
}
310429

430+
// Same, for the provisioning resolve: a green from a check that cannot fire is
431+
// indistinguishable from the green this gate printed before it existed.
432+
const provisioningControl = selfTestProvisioningUse();
433+
if (provisioningControl.failures.length) {
434+
console.error("check-platform-checklist: the provisioning-resolve check's own positive control FAILED — a `use` that resolves to nothing would pass, which is the exact defect this check was added to close.\n");
435+
for (const f of provisioningControl.failures) console.error(` ✗ ${f}`);
436+
process.exit(1);
437+
}
438+
311439
if (!existsSync(RUNNER_FILE)) {
312440
console.error(`check-platform-checklist: missing ${RUNNER_FILE} — the trap vocabulary lives in its "${TRAP_HEADING}" table and \`traps\` has nothing to validate against.`);
313441
process.exit(1);
@@ -336,6 +464,8 @@ if (files.length === 0) {
336464

337465
const allIds = new Map(); // id -> file
338466
const allItems = [];
467+
let recipeTotal = 0; // area-level provisioning recipes, across all areas
468+
let recipeRefs = 0; // item references that resolved to one
339469

340470
for (const file of files) {
341471
let doc;
@@ -353,6 +483,13 @@ for (const file of files) {
353483
continue;
354484
}
355485

486+
// Area-scoped: the universe a `provisioning.use` resolves against is THIS
487+
// file's recipe keys, so it is read once here rather than in the post-loop
488+
// cross-file section where `supersededBy` (whose universe is every id in the
489+
// ledger) has to live. Same reporting: `err(file, item.id, …)`.
490+
const recipeKeys = areaRecipeKeys(doc);
491+
recipeTotal += recipeKeys.length;
492+
356493
for (const item of doc.items) {
357494
const id = typeof item.id === 'string' ? item.id : '<no id>';
358495
const where = (msg) => err(file, id, msg);
@@ -391,6 +528,10 @@ for (const file of files) {
391528

392529
for (const msg of trapProblems(item, TRAPS)) where(msg);
393530

531+
const useProblems = provisioningProblems(item, recipeKeys);
532+
for (const msg of useProblems) where(msg);
533+
if (item.fixtures?.provisioning !== undefined && useProblems.length === 0) recipeRefs++;
534+
394535
if (item.status === 'retired') {
395536
if (typeof item.retiredReason !== 'string' || !item.retiredReason) where('retired items must carry "retiredReason"');
396537
} else {
@@ -585,5 +726,7 @@ const total = allItems.length;
585726
const active = allItems.filter(({ item }) => item.status === 'active').length;
586727
console.log(
587728
`check-platform-checklist: OK — ${files.length} areas, ${total} items (${active} active); coverage: ${mappedCount} kinds mapped, ${waivedCount} waived;` +
588-
` traps: ${TRAPS.size} documented, ${usedTraps.size} in use (extractor control: ${trapControl.checked} assertions).`,
729+
` traps: ${TRAPS.size} documented, ${usedTraps.size} in use;` +
730+
` provisioning: ${recipeTotal} area recipes, ${recipeRefs} item references resolved` +
731+
` (self-checks: ${trapControl.checked} trap-vocabulary + ${provisioningControl.checked} provisioning-resolve assertions).`,
589732
);

0 commit comments

Comments
 (0)