Repository navigation
feat(spec,core,cli)!: a scenario's requires is checked before it runs — unmet params or services skip it with a reason; requires.plugins retires into requires.services - #20511
Conversation
… skip with a reason; retire requires.plugins (WIP) Ruling B on the qa-runner family's requires key: requires.params is judged against the environment of the process running os test, requires.services (new, closed over CoreServiceName) against the target's discovery services map, and an unmet entry makes the scenario SKIPPED with its reason. The retired requires.plugins is a retiredKey() tombstone whose prescription names requires.services. Claude-Session: https://claude.ai/code/session_01ARcDurZ5j34RdqsGgc4jgH Co-authored-by: Claude <noreply@anthropic.com>
… and the plugins refusal; changeset Claude-Session: https://claude.ai/code/session_01ARcDurZ5j34RdqsGgc4jgH Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ARcDurZ5j34RdqsGgc4jgH Co-authored-by: Claude <noreply@anthropic.com>
…the plugins tombstone Claude-Session: https://claude.ai/code/session_01ARcDurZ5j34RdqsGgc4jgH Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 3 package(s): 2 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 147 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 74ed016dfacf857e31425a811ad38eee063f7dde && git checkout 74ed016dfacf857e31425a811ad38eee063f7dde
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 4a1df19656bf5b12258ef20e5aa9e8d99c3461bd 75d17a115ee806c21e916caf4c3c3cbce7e32e56 && git checkout -B drift-repro 4a1df19656bf5b12258ef20e5aa9e8d99c3461bd && git merge --no-ff 75d17a115ee806c21e916caf4c3c3cbce7e32e56
node scripts/docs-audit/affected-docs.mjs --json 4a1df19656bf5b12258ef20e5aa9e8d99c3461bd
|
Contract reviewServed-tier: Inputs: card #20289 (body and all 12 comments — triage ① Derived judgmentsRuling B, parameter by parameter — each executed, each right.
The five pins — each present and load-bearing.
"One reader over one existing contract, no new served surface" — verified on the diff. Divergence —
Retirement route against the Ablation and lint — judged from the code. Flipping Texts this PR falsifies — corrected, each true after the PR. DELIBERATE CORRECTION —
merge-tree. ② Semver level
Clause-②: yes (narrowing) Note for the seat: on this head the ③ Boundary flagsDev flags (
Check-runs on Implemented-by: VERDICT: PASS |
Fixes #20289
Clause-②: yes (narrowing)
BREAKING —
scenarios[].requires.pluginsis removed (aretiredKey()tombstone whose refusal namesrequires.services). Shipped asminorunder the launch-window convention; the ADR-0087 disposition isregistered qa-scenario-requires-plugins-retired, carried by the adr-0087 marker in.changeset/20289-requires-services-skip.md(the filecheck:adr-0087-registrationreads).Executes ruling B on the card (
5863822176, batch 232, item 1, maintainer 「同意」): the fifth and last key of theqa-runnerfamily. The four sibling keys landed in PR #20341; this PR completes the card.What changes
@objectstack/spec(qa/testing.zod.ts)requires.params— judged against the environment of the process runningos test: each variable must be set and non-empty (an unconfigured CI secret arrives as an empty string, so empty counts as unset).requires.services— NEW,z.array(CoreServiceName): each key must be declared by the target's discovery document asenabledwithstatus === 'available'(ADR-0076 D12). A misspelled key is refused when the suite loads.requires.plugins—retiredKey()tombstone. The prescription namesrequires.servicesand carries the plugin → service mapping, derived fromCORE_SERVICE_PROVIDER(the provider table both discovery builders read) so it cannot name a package that does not fill the slot. This answers the ruling's confidence gap for out-of-repo authors.RETIRED_KEYS_BY_MAJOR[18]gainsqa/TestScenario:requires.plugins; D3 semantic entryqa-scenario-requires-plugins-retired; no D2 conversion (a QA suite is a loose JSON fileos testloads, never a stack collection member or a stored row — therest-api-config-dead-keys-retiredprecedent).qa.scenarios.requiresdead→live(evidencerunner.ts#judgeRequirements,test.ts#summaryLine; producerhttp-adapter.ts#readTargetServices+ the discovery builders, andtest.ts#runfor the env).state-counts.md:qa8/1 → 9/0. README row updated.@objectstack/core(qa/)TestRunnerjudgesrequiresbefore the first step, setup included. Every unmet entry is reported (params first, then services); a service reason lists what the target declares available.TestResult.status: 'passed' | 'failed' | 'skipped'and, on a skip,skipped: { reason, unmet[], availableServices? }.passedstays,falseon a skip.TestExecutionAdapter.readTargetServices?()— optional.HttpTestAdapteranswers it from the SAME memoised discovery probe its record actions use: the document is now kept instead of read forroutes.dataand dropped. A suite that requires no service issues no extra request (anapi_call-only suite still probes nothing). An adapter without the method skips a service requirement rather than running it.new TestRunner(adapter, { env })— optional, defaults to this process's environment.@objectstack/cli(os test)⏭️ Scenario: NAME [ID] (skipped)with aSkipped:reason line.SUCCESS: 3 scenarios passed. 1 skipped (not run, not counted as passed).With nothing skipped the summary lines are byte-identical to before.No scenario ran: …instead ofSUCCESS, exits 0, and exits 1 under--fail-on-empty(description and flag help updated).Measured before building (the dispatch's mechanism assumptions)
HttpTestAdapterissuesGET {apiBase}/discoveryat most once per adapter (memoised promise), andos testbuilds one adapter per run. It was LAZY: fired only by the first record action, and it keptroutes.dataonly. Both producers (metadata-protocolgetDiscovery,runtimegetDiscoveryInfo) emitservices[name].{enabled,status}keyed byCoreServiceNamemembers only. So no fetch is added: therequires.servicesjudgement is a second trigger of the same memoised probe, and a run still issues at most one discovery request (pinned: 4 CLI runs against a stub → 4 discovery hits).requiresread nowhere onmain, ledger rowdead— HOLDS (git grepoverpackages/core/src/qaandpackages/cli/src: zeroscenario.requiresreads;liveness/qa.jsonrowdead).--fail-on-emptytoday fails exactly two things: a glob that loaded no suite, and a--tagsselection that selected zero scenarios. DESELECTED scenarios never reach the runner and appear only in the--tags … selected N of M; K deselectedline; a SKIPPED scenario was selected, reached the runner and was refused by its own precondition, so it prints its own line and reason and is counted on the summary. The all-skipped case joins the same posture as the other two.requiresis a non-strictz.object()⇒retiredKey()tombstone (a bare deletion would strip it silently). ADR-0087 for a key nested atscenarios[].requires.plugins: oneRETIRED_KEYS_BY_MAJOR[18]entry spelledqa/TestScenario:requires.plugins(nested key of an inline block, noauthorable-surface/line of its own — theapi/RestApiConfig:documentation.enabledprecedent) plus one D3 semantic entry; no D2 conversion. The liveness walk classifiesrequiresas one block, so the tombstone needs no row.Pins (the ruling's five, plus the kit)
servicesentry skips and lists the declared servicescore/src/qa/runner.test.ts(structuredunmet+availableServices),cli/test/qa-requires-skip-run.test.ts(printed reason)paramsentry skips naming the variable--fail-on-emptycli/test/qa-requires-skip-run.test.ts(noSUCCESS, exit 0; exit 1 strict)pluginskey is refused at parse with the prescriptionspec/src/qa/testing.test.ts(code, path, prescription,tscnever),cli/test/qa-suite-schema-load.test.ts(refused atos testload)Plus:
spec/src/qa/requires-plugins-retirement.test.ts(ADR-0087 registration + the tree-scoped absence pin overpackages/examples/skills/content/scripts, inside the radius@objectstack/specalready declares; registered invitest.repo-tests.json),core/src/qa/http-adapter.test.ts(one probe answers both questions; failure readings),cli/test/qa-requires-summary.test.ts(summary line shapes).Verification
All code runs at head
75d17a11(the final commit; the only later-than-code commit regeneratedtesting.mdx, and the targeted pass ran on that same tree before committing it). Suites and builds ran underos-verify-lock; lock seconds are shared-box figures.@objectstack/spectest(local):Test Files 572 passed (572)·Tests 16797 passed | 1 todo (16798).@objectstack/spectest:repo:Test Files 39 passed (39)·Tests 694 passed (694)(includes the new tree-scoped pin).@objectstack/coretest:Test Files 59 passed (59)·Tests 1555 passed (1555);src/qaalone69 passed.@objectstack/cli, theos testfiles — unit tier (qa-suite-schema-load,qa-requires-summary,qa-tags-selection,vitest-tiers-partition):46 passed; integration tier (qa-requires-skip-run,qa-names-and-tags-run, spawningbin/run-dev.jsagainst anode:httpstub):15 passed. The rest of the cli suite is declared to CI (pnpm testruns both tiers);qa-empty-glob-exit-code.e2e.test.tswas named and matched no vitest project in this package — NOT MEASURED here, reason: not a member of either project.typecheckexit 0 for@objectstack/spec,@objectstack/core,@objectstack/cli(each includingcheck:test-typecheckOK). Reverse check of the type channel: the@ts-expect-erroron an authoredrequires.pluginsinspec/src/qa/testing.test.tsis consumed (a stale.d.tswould leave it unused and red), and core/cli tests typecheckrequires.services, a key only the rebuilt.d.tscarries.check:generated: first run✗ 1 of 15 artifact(s) stale(check:docs);--fixregeneratedcontent/docs/references/qa/testing.mdx; re-run✓ All 15 generated artifacts are up to date.check:liveness:qa 9 classified (live 9);✓ packages/spec/liveness/state-counts.md is current.scripts/ablation-replace.mjs, committed tree):runner.tsanchorif (unmet.length === 0) return undefined;flipped to a greater-or-equal-zero comparison (never skip). Anchor 1 → 0, blob77e3f4c5a76c→5df21094f9fd.runner.test.ts:7 failed | 23 passed (30)— exactly the seven skip-asserting pins; the met-service CONTROL, the no-probe pin, thefailed-status pin and the 20 pre-existing tests stayed green. Expected direction: red; observed: red. Restore: blob77e3f4c5a76c== HEAD,git diff HEADempty. Nodistleg: the subject is imported relatively fromsrc.dispatch-gates --commands --repo objectstack-ai/objectstackat75d17a11derived 122 families; all 122 ran with exit codes recorded to disk;--ran:122 derived, 122 run, 0 NOT-MEASURED, 0 UNRUN. 120 exit 0. Four first answered exit 3 (PREREQUISITE NOT MET:check:skill-examples,check:dual-build-cjs-loads,check:i18n-coverage,check:type-check-debt) and exit 0 after their named prerequisites were built. Two exit 1, neither this diff's:check-empty-changeset --base origin/main— the DELIBERATE CORRECTION of the pending note (section below). Expected red.check:platform-checklist—areas/identity-auth.json: ABSENT SYMBOL packages/plugins/plugin-auth/src/auth-plugin.ts#twoFactor. The same single problem at the basefc0db22band at currentmain2b24b8b8(control runs in a detached worktree); this diff touches neither file.eslint.config.mjsitself: 16 linted by the config, 9 ignored (.md/.mdx/.json).eslint --no-inline-config --format jsonover the 16: 16 results, 0 errors, 0 warnings.parserOptions.project/projectServiceare null for all 16 (no type-aware linting), so no untouched file's verdict can move. The repo-widepnpm lintis CI's.merge-treeof this head ontomain2b24b8b8is clean (the one shared file,migrations/registry.ts, is disjoint prose on main's side).mainis not merged in; CI validates the merge ref.Texts this PR would otherwise falsify, corrected in the same change
content/docs/deployment/cli.mdx§os test: saidrequiresis not checked; now documents both keys, the skip line and the exit posture.docs/qa/platform-checklist/areas/cli.json(cli.qa-suite-execution): knownGap and theqa.jsonsource note; revision 5 → 6.packages/spec/scripts/liveness/check-liveness.mtsheader comment: namedscenario.requiresas the one key still unread..changeset/20289-os-test-names-tags.md(PR feat(qa): os test prints suite and scenario names and selects scenarios by --tags #20341's, not yet released) ends with a@objectstack/specbullet sayingTestScenario.requires"is still checked by nothing … a scenario that declares a plugin the target lacks still runs". This PR makes that false in the same release, so the bullet now says the ledger moved the four keys toliveand thatrequiresis checked in this release under its own note. Nothing else in that note changed. This is the DELIBERATE CORRECTION classcheck-empty-changeset.mjsnames:Check Changesetis expected to stay RED on that row (it is not a required context),skip-changesetis not applied, and the correction needs a person's confirmation on this PR.Acceptance notes
requires.servicesis closed overCoreServiceName. The ruling says "array of service keys"; both discovery producers keyservicesbyCoreServiceNamemembers only, so the closed enum loses nothing and turns a misspelling into a parse refusal instead of a runtime skip — the contract-tightening direction. The ruling's "a misspelling skips loudly" still holds for a valid key the target does not declare. If the seat reads the ruling as an open vocabulary, it is a one-line change toz.array(z.string()).TestResult.passedis kept (falseon a skip) beside the newstatus. A consumer that counts!passedas a failure reads a skip as a failure — loud, never a silent pass; the changeset tells it to readstatus.scenariosis[](no--tags, nothing skipped) still printsSUCCESS: All 0 scenarios passed.and exits 0 under--fail-on-empty— the all-skipped posture does not widen to it.@objectstack/coreand@objectstack/cliareminor(additive public fields and an optional interface method; new output and exit posture);@objectstack/specisminorwith the BREAKING banner. All three share the changesets fixed group.Generated by Claude Code