Repository navigation
test(drivers): a conformance run that discovers zero drivers is a failure, not an OK (#4646) - #4648
Merged
Merged
Conversation
…lure, not an OK (#4646) check-driver-conformance discovers driver packages from disk under a hardcoded DRIVERS_DIR. listDir swallows ENOENT and returns [], and all three invariants iterate the discovered set — CONSUMED over `drivers`, RECONCILED over LEDGER (empty since #4405, the intended steady state), CLASSIFIED not over drivers at all. A stale DRIVERS_DIR therefore printed `OK — 0 covered cell(s)` and exit 0. CI never had this exposure: lint.yml runs `pnpm check:driver-conformance`, which is `--self-test && audit`, and the self-test carried a discovery assertion. The false green was on the bare `node scripts/check-driver-conformance.mjs` the script's own header documents as a usage. Leaving the guard there was wrong twice over. It read `drivers.length >= 3 && drivers.includes('driver-sql')` — a hardcoded name and count inside the one script whose stated rule is that drivers come from disk and are never listed, so both wanted hand-editing on the next driver added or package moved, which is when the guard earns its keep. And its failure text named neither DRIVERS_DIR nor the stale path. DISCOVERED is now a fourth invariant in audit(), and its message names the directory searched. The self-test drives the invariant in both directions instead of standing in for it, and asserts nothing about which drivers exist. The case-set axis cannot rot this way and is left alone: CASE_SETS is a declared expectation, so a vanished spec/src/data fails CLASSIFIED's reverse direction with one error per case-set. The driver axis is disk-discovery with nothing declared to reconcile against — that asymmetry is why zero was reachable on one axis and not the other, and it is what DISCOVERED supplies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VTj56JLhVk585TA8Ez3yGK
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
size/s
documentation
This was referenced Aug 2, 2026
os-zhuang
marked this pull request as ready for review
August 2, 2026 14:30
This was referenced Aug 15, 2026
akarma-synetal
pushed a commit
to akarma-synetal/framework
that referenced
this pull request
Oct 7, 2026
…ned and URL-spelled citations, pinned in one spelling table (objectstack-ai#20989) Fixes objectstack-ai#20636 Clause-②: no The family closeout for the citation extractor in `scripts/check-issue-citations.mjs`: every spelling a seat measured as invisible to the diff gate and the census is now read, every exclusion keeps only the shapes it was measured protecting, and the self-test carries the one enumeration table the triage asked for (48 spellings, each "extracted as" or "not a citation, because"). Landing site: `scripts/check-issue-citations.mjs` only. `scripts/check-doc-authoring.mjs` is untouched; H2 below says why. ## What changed - **Hyphen after the number.** The lookahead no longer refuses a `-`, so `objectstack-ai#13398-class`, `objectstack-ai#5347-A` and `ui#6206-B` read as citations of their number. It still refuses a word character, so a hex colour stays out. - **Slash before the `#`.** A `/` is now valid context before a bare `#`. It stays refused only before a qualifier candidate, so a URL path fragment such as `https://example.com/docs/page#12` never reads `page` as prose. A `/` right after a digit is the exception, so `objectstack-ai#3076/objectui#2614` still reads its own qualifier. - **Slash-joined continuation.** In `#A/#B`, the second number takes the chain head's reading: bare after a bare or prose head, the head's repository after a qualified head, and an ordinal after an ordinal. Only a joined `/` continues a chain. `objectui#1 / objectstack-ai#2` and `objectui#1 + objectstack-ai#2` stay two separate readings. - **URL spelling.** `https://github.com/OWNER/REPO/issues/N` and `.../pull/N` are now citations, qualified by their own `OWNER/REPO`. That puts `objectstack-ai/framework` in this repository and makes every other repository's URL cross-repo, never a finding. Inside a markdown link `[#N](URL)`, the citation counts once. - **Head rows.** I retired `re-charter`, `clause` and `option` (named in the thread), plus `acceptance` and the section mark (found by the same measurement). Each protects 0 sites repo-wide at two or more digits and hides board citations. The grammar's two-digit floor already keeps one-digit ordinals out. I added a `](` row, the narrower guard the hyphen exclusion leaves behind for markdown in-page heading anchors. - **Self-test.** - A new `spellings` battery (56 cases) reads the table row by row. - Every head row must excuse at least one table row, and every required spelling (the card's list plus the thread's) must be present. - The `live-corpus` battery gains three floors, one per new arm, each counting only a number spelled once on its line. - The roster floor rises from 6 to 9 batteries. - Total: 114 cases in 8 batteries became 173 in 9. - I edited the header in place and kept its line count, because `scripts/pm/dispatch-gates.mjs`'s self-test pins `scripts/check-issue-citations.mjs:204 local-env`. The marker is still on line 204, and that pin passes (see Gates). ## H1: what each exclusion protected and hid Measured on `3693a1b50` over the declared surfaces. Every arm was judged against one enumerated board: 188 pages, frontier 20959, 2026-09-30T22:55Z. The instrument mirrors the gate's extractor and matched it file for file on all 2,640 files (0 mismatches). | exclusion | hid (sites, dead) | protected in the declared surfaces | disposition | |---|---|---|---| | hyphen after the number | 58, 1 dead (8 cross-repo) | 0: no numeric range, slug or hex-like token. Repo-wide: in-page heading anchors, 16 lines in `docs/design/**` and `skills/**`, plus one range, `docs/audits/...md`, whose first number is a citation | dropped; the `](` row keeps the anchor out | | `/` before the `#` | 528, 10 dead: 523 `#A/#B` second numbers and 5 `TOKEN/#N` such as `ADR-0049/objectstack-ai#1888` | 0 paths and 0 URL fragments | narrowed to the candidate arm; continuations read as their chain | | head `re-charter` | 0 left on this tree (the 26 dead `re-charter objectstack-ai#13135` were rewritten by PR objectstack-ai#20750) | 0; only the gate's own fixtures used it | retired | | head `clause` | 0 in the surfaces; 4 in deferred test files, all board citations | 0 | retired | | head `option` | 1 (`option objectstack-ai#14088`, live); 1 more under `scripts/**` | 0 | retired | | head `acceptance` | 1 (`the silent acceptance objectstack-ai#6132 closed`, live) | 0 at two or more digits | retired (in-place, below) | | head section mark | 0 in the surfaces; 4 in test files (`§6 objectstack-ai#11176's decisions`) | 0 at two or more digits | retired (in-place, below) | | heads kept | none measured hiding a citation | directive 203 (max 13), `PD` 85 (max 13), batch 276 (69 distinct, 11 to 227), `OQ` 10, `PKCS` 1 | kept | The 8 slash chains headed by another repository are not a case where the two populations cannot be told apart. The 5 on objectui's public board each name objectui's record, the issue and then the pull request that fixed it, read one by one against both boards: - `objectui#2715/objectstack-ai#2717` - `objectstack-ai#2711/objectstack-ai#2722` - `objectstack-ai#2725/objectstack-ai#2732` - `objectstack-ai#2967/objectstack-ai#2904` - `objectstack-ai#4648/objectstack-ai#4901` This repository's records with the same numbers are unrelated. The other 3 (`cloud`, `hotcrm-heimao`) are boards one credential cannot read, so they stay unjudged, as they were before. ## H2: where the URL spelling belongs The extractor. At `3693a1b50`, 57 URL sites sit in the gate's projection: 56 in package comments and 1 link on a release page. 4 of them are dead. Only 2 URL sites in package sources are inside string literals, both internal `note:` strings in `packages/runtime/src/route-ledger.ts`. `check:doc-authoring` asks a different question: may a runtime string carry a tracker reference at all? It reads string literals, skills and spec refusal messages. The two projections are disjoint, so adding the URL to the extractor double-counts nothing there. Inside the extractor, the one double-spelled site (`[objectstack-ai#15325](...objectstack-ai/issues/15325)` on `v17/17-3.mdx`) counts once. `check-doc-authoring.mjs` is not touched. ## H3: open PRs' added lines All 13 open PRs at 2026-09-30T23:2xZ: their heads were fetched into a private ref namespace (deleted afterwards). For each, the BASE extractor and this one were run over the lines it adds, against its merge base. Result: 85 added-line citations under both extractors, 0 newly extracted, 0 lost. No PR's verdict changes. Lines a PR does not add are never judged, which is unchanged and pinned in the `diff-scope` battery. ## Census, before and after The gate's own `--census --json`, once with the `3693a1b50` script and once with this one, over the same tree: | | judged | resolves | resolves as PR | cross-repo | allocated-but-absent | |---|---|---|---|---|---| | before (frontier 20965) | 37,152 | 33,403 | 1,985 | 1,024 | 740 | | after (frontier 20966) | 37,796 | 33,917 | 2,083 | 1,041 | 755 | That is 644 more judged sites and 15 more dead ones, with 0 findings lost. By arm: hyphen 1 dead, slash 10 dead, URL 4 dead. Newly visible dead sites per lane. I rewrote none of them; they belong to the lane cards: - **objectstack-ai#20594 (`domain:cli`): 1.** `packages/rest/src/rest-server.ts:7456`, objectstack-ai#11006. - **objectstack-ai#20595 (`domain:engine`): 6.** - `driver-sql`: `sql-driver.ts:3933` (URL, objectstack-ai#17590), `:12110` (objectstack-ai#10629), `:16106` (objectstack-ai#17343). - `metadata`: `loaders/ambiguous-metadata-stem.ts:40` (objectstack-ai#14423). - `metadata-protocol`: `migrations/partial-index-probe.ts:395` (objectstack-ai#16657). - `objectql`: `plugin.ts:1496` (objectstack-ai#10629). - **objectstack-ai#20596 (`domain:services`): 0.** - **objectstack-ai#20597 (`domain:spec`, `packages/lint`): 0.** - **objectstack-ai#20234 (`packages/spec/src`): 7.** - `data/datasource.zod.ts:701` (objectstack-ai#9040). - `data/filter.zod.ts:1040` (URL, objectstack-ai#17590) and `:1042` (URL, objectstack-ai#17286). - `data/value-roundtrip-conformance.ts:100` (URL, objectstack-ai#12380). - `ui/component.zod.ts:593` and `:669` (objectstack-ai#6276), and `:3771` (objectstack-ai#9972). - **Release pages (`domain:devx`, no lane card): 1.** `content/docs/releases/v17/index.mdx:168` (objectstack-ai#6075). This census was run on the PR head's tree, not after landing. A re-run after landing reads the same corpus plus whatever `main` has gained by then. ## In-place fixes beyond the three named rows I retired the `acceptance` and section-mark rows here rather than filing them. All four conditions hold: 1. They are the same defect class as `option` and `clause`: a head row hiding a board citation. 2. The fix is mechanical, and the shape is pinned by the table. 3. The file is this claim's own surface (`NON_CITATION_HEADS`). 4. The same gate's self-test covers them, so no new verification surface is added. Evidence: `acceptance objectstack-ai#6132` (live) in `packages/formula/src/cel-pushdown-limits.ts:82`, and the section-mark sites in deferred test files. Neither row has any two-or-more-digit ordinal anywhere in the repository. ## Gates (final head `25d96fcc0`) I re-derived the list with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, which gave 32 commands. The same derivation on a throwaway tree at `origin/main` (`05be35259`) with this diff applied gave an identical list. I ran all 32, plus `pnpm check:doc-authoring` and the self-test, and every one exited 0. Reconciliation verdict line: `✓ dispatch-gates --ran: 32 derived famil(ies) accounted for — 32 run, 0 NOT-MEASURED (a DERIVED zero — all 32 recorded an exit code and none of them is 3).` Verdict lines worth quoting: - `node scripts/check-issue-citations.mjs --self-test`: `✅ ... every spelling enumerated ... (173 cases, 9 batteries)`. It also passes on `origin/main` `05be35259` with this diff applied. - `node scripts/check-issue-citations.mjs` (diff mode): `✅ check-issue-citations: no issue citations added against 3693a1b (0 file(s) read).` The script lives in the deferred `scripts/**` surface. - `pnpm check:pm-dispatch-gates`: `✓ dispatch-gates self-test: 1976 cases pass.` (1237.9s). Its pin `scripts/check-issue-citations.mjs:204 local-env` holds. - `pnpm check:doc-authoring`: `✓ doc authoring guard: ... hold the baseline — 549 pinned site(s)`. - `pnpm check:nul-bytes`: `check-nul-bytes: OK (scanned 9582 text file(s) ...)`. - `node scripts/check-scripts-symbol-anchors.mjs`: `✅ ... 3706 anchors across 282 scripts resolve`. ## Ablations (one-shot, from committed `25d96fcc0`, via `scripts/ablation-replace.mjs`) Every leg was expected to go red, and every leg did. Each one restored to blob `c732ce2e21c7` (equal to HEAD) with an empty `git diff HEAD`. No permanent ablation file is left. | mutation | first red | |---|---| | hyphen refused again after the number | `the live corpus must yield a #N-word citation` | | `/` refused again before the `#` | `the live corpus must yield a slash-joined #A/#B second number` | | URL arm disabled | `the live corpus must yield a URL-spelled citation` | | `option` head row restored | `spelling option #N in "the option objectstack-ai#14088 gave" must read objectstack-ai#14088; got nothing` | | continuation disabled | `spelling repo#A/#B ... must read objectstack-ai/objectui#2711, objectstack-ai/objectui#2722` | | `](` row disabled | `a markdown link's in-page heading anchor is not a citation` | | link de-duplication disabled | `spelling [#N](URL) ... must read objectstack-ai#15325` | The first slash ablation, run before the floors were tightened, went red in the table but left the live-corpus continuation floor green. A line citing the same number twice let a plain citation stand in for the slash-joined one. Commit `25d96fcc0` makes each new floor count only a number spelled once on its line. The re-run above is red at that floor. ## Acceptance notes - **URL spelling in runtime strings.** `check:doc-authoring` does not read it. At `3693a1b50` the population is 2 internal route-ledger `note:` strings (`packages/runtime/src/route-ledger.ts:459`, `:465`), and no author-facing door shows them. Noted, not filed; carrier: none. - **Range second numbers.** In a range such as `objectstack-ai#712-714`, the second number carries no `#` and is not read. There is 1 site repo-wide, in `docs/audits/**`, outside the declared surfaces. Pinned in the table with its reason. - **Dormant test-file citations.** The 8 test-file citations the retired `clause` and section-mark rows hid are in the deferred test surface. They become visible only when that surface is swept. - **Line-number pin.** `dispatch-gates.mjs` pins this file's `local-env` marker by line number (`:204`), so any future header growth above it has to move that pin in the same PR. No changeset: a root `scripts/` file publishes nothing (the root `package.json` is private, and no package's `files` ships `scripts/`), so this PR takes `skip-changeset`. --- _Generated by [Claude Code](https://claude.ai/code/session_017VaLJnYwhPsanVCe9dMCJU)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4646.
问题
scripts/check-driver-conformance.mjs从磁盘发现 driver 包，根目录写死在DRIVERS_DIR。listDir吞掉 ENOENT 返回[]，而三条不变量全部遍历「发现到的集合」：drivers→ 零次迭代LEDGER→LEDGER自 filter-logic conformance 少了两个 backend：MongoDB 的 translateFilter 从来没被 #3774 的标准收编 #4405 起为空（注释写明「the intended steady state」）于是一个失效的
DRIVERS_DIR会打印OK — 0 covered cell(s)并 exit 0。先把范围说准：CI 从来没有这个暴露。
lint.yml:304跑的是pnpm check:driver-conformance，即--self-test && audit，self-test 里有一条 driver 发现断言会先失败。假绿只出现在脚本 header 第 21 行文档化的node scripts/check-driver-conformance.mjs裸调用上。为什么不是「self-test 已经管了，就这样吧」
那条守卫本身有两个问题：
drivers.length >= 3 && drivers.includes('driver-sql')—— 一个硬编码的 driver 名和数量，出现在唯一一份声明「driver 从磁盘发现、绝不列表」（:145）的脚本里。下次加 driver 或挪包时两处都要手改，而那正是守卫该发挥作用的时刻。discovers driver packages from disk既不提DRIVERS_DIR也不说路径已失效，踩中的人得自己去找。改动
audit()增加第四条不变量 DISCOVERED，信息里带上实际搜索的目录：检查体抽成
discoveredErrors(drivers)，好让 self-test 驱动这条不变量本身而不是它的替身；self-test 里对 driver 名和数量的断言随之删除。为什么不给 case-set 轴加同样的检查
它不会这样烂掉，实测过。
CASE_SETS是脚本里声明的预期，所以spec/src/data消失时 CLASSIFIED 的反向对账会逐条报错：driver 轴是纯磁盘发现、没有任何声明预期可对账 —— 这个不对称正是「零」只在一条轴上可达的原因，也正是 DISCOVERED 补上的那一块。给 case-set 轴再加一层是冗余。
验证
audit--self-testDRIVERS_DIR失效 →auditDRIVERS_DIR失效 →--self-testCASE_SETS_DIR失效 →auditpnpm check:driver-conformance（self-test + audit）全绿，eslint干净。失效场景通过复制脚本改常量的方式复现，未改动仓库内容。来由
来自 turso driver 归属评估（#4645）：那次重组要把
packages/plugins/driver-*挪到packages/drivers/*，正好会动DRIVERS_DIR和 self-test 里那两个硬编码值。但这个缺陷独立于重组成立，所以单独出 PR，作为 #4645 的前置项。分支名不走
claude/issue-<n>-<slug>约定，因为本会话被指派了固定分支claude/turso-driver-evaluation-beoewn。Generated by Claude Code