Repository navigation
check-driver-conformance counts a comment-only mention as coverage — a driver's own source file is scored as a conformance suite #12135
Description
Activity
PM note (
domain:devxseat, sessionaf22b339-91b5-5814-8b9b-2453fc5b3f68) — hot-file serial queue, not dispatched this batch.Considered for this batch and held. PR #12134 (fixes #12014) is open and rewrites the one file this card's remedy touches —
scripts/check-driver-conformance.mjs, +637/−17, and that is its entire declared surface ("Held to the declared surface:scripts/check-driver-conformance.mjsonly"). Any of the card's three remedies editsconsumes()in that same file. Dispatching now would put two agents in one file and produce a conflict whose resolution risks dropping one side's measurements.The card stays
pm:queue— it is not logically blocked, and its premise does not depend on #12134. This is a file-contention hold only.Release condition: PR #12134 merges (or closes). At that point the next seat should re-verify the premise on the merged ref before dispatching, because #12134 lands a large change to the same function's neighbourhood — in particular it routes the dialect axis through
scripts/js-comment-mask.mjsfor exactly the "a comment must not count" reason this card is about. That may make remedy (2) already partly present, or may change which remedy is cheapest. Upstream merged ≠ this card still stands: re-readconsumes()on the merged tree first.No action needed from #12134's author — this is a note to whoever picks this card up.
Generated by Claude Code
Claim —
domain:devx @ objectstackseatClaim:sessionaf22b339-91b5-5814-8b9b-2453fc5b3f68· branchclaude/issue-12135-coverage-scan-comment-mask· worktree../objectstack-12135· model tier opus.Declared file surface:
scripts/check-driver-conformance.mjs— the coverage scan (coveringFiles/drivenFrom/consumes) and its--self-test. ⛔ Not the dialect axis #12134 just landed, ⛔ notscripts/js-comment-mask.mjs, ⛔ no driver test files.Clause ②: no. A census detector; no shipped accept/reject door moves.
Unblocked by: PR #12134, merged
dd391cee42. It was the open claimant on this file.⭐ Premise re-verified on the merged ref — it survives, and #12134 sharpened it
This seat held the card specifically because #12134 routes the dialect axis through
js-comment-mask.mjsfor the same "a comment must not count" reason, and might have pre-empted the remedy. Measured onorigin/mainrather than assumed — it did not:// scripts/check-driver-conformance.mjs :116 import { stripComments } from './js-comment-mask.mjs'; // imported… :971 if (drivenFrom(readFileSync(file, 'utf8'), marker, …)) // …but the COVERAGE scan is RAW
consumes()delegates tocoveringFiles(), so "is this cell covered" is still answered from unmasked text, comments included. The card stands exactly as filed.⭐ And #12134's own prose is now the strongest argument for this card. Immediately below that raw read, the file it just landed says:
The stance is read off source TEXT, so a MENTION must not count as a declaration — and on this tree the mentions are everywhere, because the files that were FIXED describe the fix in prose. …
sql-driver.tsnamesFILTER_LOGIC_CASESandclient: 'postgres'in comments only. Measured: 5 of this driver's 8 covering files change their client-literal answer between raw and masked text.So the principle is written down, the measurement is already done, and
sql-driver.ts— the card's own example of a driver source file scored as a conformance suite — is named in it. The two axes now disagree with each other inside one file: the dialect axis refuses a mention, the coverage axis accepts one.⚠️ Note the import isstripComments, notmaskComments. Whether the coverage scan wants the same one is a question for the dev — the two differ in whether offsets are preserved, and the coverage scan reports files rather than line numbers.
Generated by Claude Code
{ "issue": 12135, "status": "done", "branch": "claude/issue-12135-coverage-scan-comment-mask", "pr": "https://github.com/objectstack-ai/objectstack/pull/12321", "premise_still_valid": true, "summary": "Confirmed the premise on my own base (15249270fb, later merged to 4879d71cc2): scripts/check-driver-conformance.mjs imported stripComments for the dialect axis only, while coveringFiles() read raw text, so consumes() answered 'is this cell covered' from unmasked prose. Routed coveringFiles's read through a new codeOf() helper using stripComments — chosen on js-comment-mask.mjs's own documented discriminator (reports a LINE or byte offset -> maskComments; feeds a scanner and reports neither -> stripComments), which this axis satisfies because it reports FILE paths only; stripComments is also the cheap form (the module records a 51x slowdown for maskComments on a package-level gate). Deliberately did NOT memoise per path: the self-test rewrites one a.test.ts path three times and asserts a different verdict each time, so a cache would make those assertions read a stale body. RAW-VS-MASKED COVERAGE-ANSWER COUNT: 0 of 45 (driver, marker) cells change; headline stays '45 covered cell(s), 0 in the DEBT ledger, 0 exempt', so nothing moves into the DEBT ledger and no published number changes. CELLS THAT MOVE: none. Two per-FILE covering answers move, both driver implementations rather than suites, both for FILTER_LOGIC_CASES: packages/drivers/driver-sql/src/sql-driver.ts and packages/drivers/driver-mongodb/src/mongodb-filter.ts. Neither cell's evidence file moves (walk order already put a real suite first), so no consumes() path quoted back to an author changes. ZONE 2.5 — the card's specific example HOLDS and I verified it rather than inheriting it: the gate itself printed 'note packages/drivers/driver-sql/src/sql-driver.ts covers a cell but is not a test file', and that note disappearing is the ONLY line of gate output this PR changes. The mongodb one was never printed (driver-mongodb is single-backend, so the dialect axis never reaches it) and was found only by measurement — PM's #12134-derived numbers named sql-driver.ts alone, so that second file is new information. Added three self-test cases plus a mirror; the first asserts the fixture IS accepted by the unmasked rule, which is what keeps the other two falsifiable against a revert.", "tests": "All at final commit 4879d71cc2 (merge of origin/main after dispatch-gates printed STALE TREE; re-derived clean afterward). Gate union: `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack` named 9 families; all 9 run under `bash scripts/pm/os-verify-lock.sh -c ...`, each exit code captured by redirecting to a file BEFORE any pipe, none narrowed, none reported PREREQUISITE NOT MET. Lock: 'os-verify-lock: VERDICT command-exit 0 · held the lock 28s · waited 0s'. Verdict lines quoted from each gate's own output: check:agent-test-spelling EXIT=0 '✓ sweep is clean'; check:cli-command-ids EXIT=0 '✓ check-cli-command-ids: 276 command-id literal(s) across 99 file(s) outside packages/cli all resolve to a real command path' (carries one PRE-EXISTING baselined violation, `objectstack publish`, tracked as #12223 — untouched by this PR, gate still exits 0); check:cross-package-test-inputs EXIT=0 'OK: 16 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.'; check:driver-conformance EXIT=0 'check-driver-conformance: OK — 45 covered cell(s), 0 in the DEBT ledger, 0 exempt.'; check:entry-guard EXIT=0 '✓ check:entry-guard: 167 scripts/ file(s) — every entry guard goes through invoked-as.mjs'; check:parse-guard EXIT=0 '✓ check:parse-guard: 166 scripts/ file(s) — every TypeScript parse goes through ts-parse.mjs.'; check:pnpm-filter-targets EXIT=0 '✓ check:pnpm-filter-targets: 136/173 --filter occurrence(s) across 28 file(s) resolve against 78 workspace package(s)'; check-ci-filter-parity.mjs EXIT=0 'OK: all 96 declared cross-package glob(s) (81 unique) are covered'; check-cross-package-test-inputs.mjs EXIT=0 'OK: 16 package(s) read outside themselves, all declared'. check:driver-conformance, check:entry-guard, check:parse-guard and check:pnpm-filter-targets each run their --self-test first, so the new cases sit INSIDE the gate union. ABLATION — direction predicted in writing before running, both legs wrapped in `trap '<restore>' EXIT INT TERM`. NO REBUILD LEG APPLIES and this is stated rather than skipped: the subject is a .mjs gate script executed directly from source by `node scripts/...`, with no package build and no dist/ between the edit and the measurement, so there is no stale-artifact path for a false green to hide in; the mutation IS the file the run reads. Mutation confirmed on disk by grep counts anchored to the exact text being changed, never by the editor's exit code and never by a bare `git diff --stat`. LEG A (revert the masking in codeOf): predicted the two new self-test cases turn red while the precondition case stays green. On-disk proof 'stripComments-form before=1 after=0 ; raw-form after=1'. Observed 'ABLATED SELFTEST EXIT=1' with exactly two failures — '#12135 — a marker mentioned only in a comment must not count as coverage' and '#12135 — and the file is not returned as a covering file either'; the precondition case stayed green as predicted. LEG B (a real covering file's reference made comment-only): packages/drivers/driver-memory/src/memory-aggregation-conformance.test.ts is the SOLE covering file for driver-memory x AGGREGATION_CASES; commented out its two live `for (const c of AGGREGATION_CASES)` lines, leaving the import and two docblock mentions — the exact shape found on sql-driver.ts. On-disk proof 'live code refs before=2 after=0 ; injected comment refs=2 ; import line kept=1'. Predicted covered -> MISSING and gate red; observed 'FIXED-DETECTOR EXIT=1' with 'x CONSUMED: driver-memory does not run AGGREGATION_CASES' and 'check-driver-conformance: 1 problem(s).'. Counterfactual on the SAME mutated bytes: 'OLD raw detector: true / NEW masked detector: false' — identical bytes, opposite verdicts, which is the defect demonstrated directly. RESTORE PROOF, both legs: `git hash-object` of each restored file equals its `git rev-parse` blob id at HEAD (check-driver-conformance.mjs b01d5f344eac06565cd41eda8fbff8093b710d90 == HEAD; memory-aggregation-conformance.test.ts e3d2e58acb41b61c1d588d720c3c07ece90cfd59 == HEAD), and `git status --porcelain` is empty. NO NARROWING DECLARED — every family the derivation named was run in full. Repo-wide `pnpm lint` was not named by the derivation for this surface and is CI's run, not one I owe.", "open_questions": [], "out_of_scope_findings": [ "filed as #12320: the import regex in drivenFrom is still unbounded, so after masking it can still assemble a match across TWO import statements — a marker imported from a LOCAL module plus any later `from '@objectstack/spec/data'` scores as shared-standard coverage (proven synthetically on masked text). This is the card's unchosen remedy (1), which #12135 would otherwise close with it. Swept after the fix: 54 (file, marker) import readings across 5 drivers x 9 case-sets, 0 differ between the unbounded regex and a per-statement read — latent, not live, so filed `finding` + `domain:devx` with no `pm:queue`, unassigned." ] }Fallbacks declared. REST is closed from this dev seat (403,
GitHub access is not enabled for this session) — the known #12123 outage — in both directions it was needed:- Dedup scan before filing check-driver-conformance: the unbounded import regex still spans two import statements — a locally-imported same-named fixture counts as shared-standard coverage #12320: the REST list endpoint returned 403, so the duplicate scan could not be completed and no empty result is claimed. The local half was done (nothing in
scripts/check-driver-conformance.mjs, no.changeset/entry records the residual). Worth a re-check at triage. skip-changesetlabel: the additivePOST .../issues/12321/labelscall returned 403, so the declared fallback was used — read → union → whole-set write via MCP. Read back and confirmed: PR fix(devx): the driver-conformance coverage scan reads code, not prose #12321 carries exactlyskip-changeset,draft: true,state: open.
Per the dispatch contract the report lands at draft-PR time; CI convergence is the PM's read, not mine. Gate status at hand-off: the 9 local families are green as quoted above; CI's own job conclusions are
in_progressfrom my side.
Generated by Claude Code
Generated by Claude Code
- Dedup scan before filing check-driver-conformance: the unbounded import regex still spans two import statements — a locally-imported same-named fixture counts as shared-standard coverage #12320: the REST list endpoint returned 403, so the duplicate scan could not be completed and no empty result is claimed. The local half was done (nothing in
os-dev-report
⚠️ Supersedes comment 5414548359 — that one carried theos-dev-reportmarker in HTML-comment form and the body sanitizer ate it after the write, confirmed by read-back, so it is invisible to a marker scan. Same report, marker as literal text. No comment-edit path exists from this seat (REST PATCH is 403, #12123), hence a repost rather than an edit.{ "issue": 12135, "status": "done", "branch": "claude/issue-12135-coverage-scan-comment-mask", "pr": "https://github.com/objectstack-ai/objectstack/pull/12321", "premise_still_valid": true, "summary": "Confirmed the premise on my own base (15249270fb, later merged to 4879d71cc2): scripts/check-driver-conformance.mjs imported stripComments for the dialect axis only, while coveringFiles() read raw text, so consumes() answered 'is this cell covered' from unmasked prose. Routed coveringFiles's read through a new codeOf() helper using stripComments — chosen on js-comment-mask.mjs's own documented discriminator (reports a LINE or byte offset then maskComments; feeds a scanner and reports neither then stripComments), which this axis satisfies because it reports FILE paths only; stripComments is also the cheap form (the module records a 51x slowdown for maskComments on a package-level gate). Deliberately did NOT memoise per path: the self-test rewrites one a.test.ts path three times and asserts a different verdict each time, so a cache would make those assertions read a stale body. RAW-VS-MASKED COVERAGE-ANSWER COUNT: 0 of 45 (driver, marker) cells change; headline stays '45 covered cell(s), 0 in the DEBT ledger, 0 exempt', so nothing moves into the DEBT ledger and no published number changes. CELLS THAT MOVE: none. Two per-FILE covering answers move, both driver implementations rather than suites, both for FILTER_LOGIC_CASES: packages/drivers/driver-sql/src/sql-driver.ts and packages/drivers/driver-mongodb/src/mongodb-filter.ts. Neither cell's evidence file moves (walk order already put a real suite first), so no consumes() path quoted back to an author changes. ZONE 2.5 — the card's specific example HOLDS and I verified it rather than inheriting it: the gate itself printed 'note packages/drivers/driver-sql/src/sql-driver.ts covers a cell but is not a test file', and that note disappearing is the ONLY line of gate output this PR changes. The mongodb one was never printed (driver-mongodb is single-backend, so the dialect axis never reaches it) and was found only by measurement — PM's #12134-derived numbers named sql-driver.ts alone, so that second file is new information. Added three self-test cases plus a mirror; the first asserts the fixture IS accepted by the unmasked rule, which is what keeps the other two falsifiable against a revert.", "tests": "All at final commit 4879d71cc2 (merge of origin/main after dispatch-gates printed STALE TREE; re-derived clean afterward). Gate union: `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack` named 9 families; all 9 run under `bash scripts/pm/os-verify-lock.sh -c ...`, each exit code captured by redirecting to a file BEFORE any pipe, none narrowed, none reported PREREQUISITE NOT MET. Lock: 'os-verify-lock: VERDICT command-exit 0 · held the lock 28s · waited 0s'. Verdict lines quoted from each gate's own output: check:agent-test-spelling EXIT=0 'sweep is clean'; check:cli-command-ids EXIT=0 'check-cli-command-ids: 276 command-id literal(s) across 99 file(s) outside packages/cli all resolve to a real command path' (carries one PRE-EXISTING baselined violation, `objectstack publish`, tracked as #12223 — untouched by this PR, gate still exits 0); check:cross-package-test-inputs EXIT=0 'OK: 16 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.'; check:driver-conformance EXIT=0 'check-driver-conformance: OK — 45 covered cell(s), 0 in the DEBT ledger, 0 exempt.'; check:entry-guard EXIT=0 'check:entry-guard: 167 scripts/ file(s) — every entry guard goes through invoked-as.mjs'; check:parse-guard EXIT=0 'check:parse-guard: 166 scripts/ file(s) — every TypeScript parse goes through ts-parse.mjs.'; check:pnpm-filter-targets EXIT=0 'check:pnpm-filter-targets: 136/173 --filter occurrence(s) across 28 file(s) resolve against 78 workspace package(s)'; check-ci-filter-parity.mjs EXIT=0 'OK: all 96 declared cross-package glob(s) (81 unique) are covered'; check-cross-package-test-inputs.mjs EXIT=0 'OK: 16 package(s) read outside themselves, all declared'. check:driver-conformance, check:entry-guard, check:parse-guard and check:pnpm-filter-targets each run their --self-test first, so the new cases sit INSIDE the gate union. ABLATION — direction predicted in writing before running, both legs wrapped in a shell trap on EXIT INT TERM that restores the file. NO REBUILD LEG APPLIES and this is stated rather than skipped: the subject is a .mjs gate script executed directly from source by `node scripts/...`, with no package build and no dist/ between the edit and the measurement, so there is no stale-artifact path for a false green to hide in; the mutation IS the file the run reads. Mutation confirmed on disk by grep counts anchored to the exact text being changed, never by the editor's exit code and never by a bare `git diff --stat`. LEG A (revert the masking in codeOf): predicted the two new self-test cases turn red while the precondition case stays green. On-disk proof 'stripComments-form before=1 after=0 ; raw-form after=1'. Observed 'ABLATED SELFTEST EXIT=1' with exactly two failures — '#12135 a marker mentioned only in a comment must not count as coverage' and '#12135 and the file is not returned as a covering file either'; the precondition case stayed green as predicted. LEG B (a real covering file's reference made comment-only): packages/drivers/driver-memory/src/memory-aggregation-conformance.test.ts is the SOLE covering file for driver-memory x AGGREGATION_CASES; commented out its two live `for (const c of AGGREGATION_CASES)` lines, leaving the import and two docblock mentions — the exact shape found on sql-driver.ts. On-disk proof 'live code refs before=2 after=0 ; injected comment refs=2 ; import line kept=1'. Predicted covered becomes MISSING and gate red; observed 'FIXED-DETECTOR EXIT=1' with 'x CONSUMED: driver-memory does not run AGGREGATION_CASES' and 'check-driver-conformance: 1 problem(s).'. Counterfactual on the SAME mutated bytes: 'OLD raw detector: true / NEW masked detector: false' — identical bytes, opposite verdicts, which is the defect demonstrated directly. RESTORE PROOF, both legs: `git hash-object` of each restored file equals its `git rev-parse` blob id at HEAD (check-driver-conformance.mjs b01d5f344eac06565cd41eda8fbff8093b710d90 == HEAD; memory-aggregation-conformance.test.ts e3d2e58acb41b61c1d588d720c3c07ece90cfd59 == HEAD), and `git status --porcelain` is empty. NO NARROWING DECLARED — every family the derivation named was run in full. Repo-wide `pnpm lint` was not named by the derivation for this surface and is CI's run, not one I owe.", "open_questions": [], "out_of_scope_findings": [ "filed as #12320: the import regex in drivenFrom is still unbounded, so after masking it can still assemble a match across TWO import statements — a marker imported from a LOCAL module plus any later from-'@objectstack/spec/data' clause scores as shared-standard coverage (proven synthetically on masked text). This is the card's unchosen remedy (1), which #12135 would otherwise close with it. Swept after the fix: 54 (file, marker) import readings across 5 drivers x 9 case-sets, 0 differ between the unbounded regex and a per-statement read — latent, not live, so filed `finding` + `domain:devx` with no `pm:queue`, unassigned." ] }Fallbacks declared. REST is closed from this dev seat (403,
GitHub access is not enabled for this session) — the known #12123 outage — in all three places it was needed:- Dedup scan before filing check-driver-conformance: the unbounded import regex still spans two import statements — a locally-imported same-named fixture counts as shared-standard coverage #12320: the REST list endpoint returned 403, so the duplicate scan could not be completed and no empty result is claimed. The local half was done (nothing in
scripts/check-driver-conformance.mjs, no.changeset/entry records the residual). Worth a re-check at triage. skip-changesetlabel: the additivePOST .../issues/12321/labelscall returned 403, so the declared fallback was used — read → union → whole-set write via MCP. Read back and confirmed: PR fix(devx): the driver-conformance coverage scan reads code, not prose #12321 carries exactlyskip-changeset,draft: true,state: open.- This comment: no edit path, hence the repost above.
Per the dispatch contract the report lands at draft-PR time; CI convergence is the PM's read, not mine. Gate status at hand-off: the 9 local families are green as quoted; CI's own job conclusions are
in_progressfrom my side.
Generated by Claude Code
Generated by Claude Code
- Dedup scan before filing check-driver-conformance: the unbounded import regex still spans two import statements — a locally-imported same-named fixture counts as shared-standard coverage #12320: the REST list endpoint returned 403, so the duplicate scan could not be completed and no empty result is claimed. The local half was done (nothing in
- added a commit that references this issue
on Aug 25, 2026 - added a commit that references this issue
on Sep 1, 2026
Found while adding the dialect axis to the census (#12014 / PR #12134). Filed unassigned, and deliberately not fixed there: the remedy changes what
CONSUMEDcounts, which is a different blast radius from adding an axis.What was measured
consumes()decides a(driver × case-set)cell is covered when a file under the package'ssrc/both imports the case-set marker from@objectstack/spec/dataand references it outside the import. The import half is this regex:[\s\S]*?is unbounded on both sides, so it does not have to match one import statement. It matches anyimportkeyword anywhere in the file, any later mention of the marker, and any laterfrom '@objectstack/spec/data'— across thousands of lines, and through comments.Measured on
origin/mainate7480a8067:sql-driver.tsis the driver implementation, not a suite. It namesFILTER_LOGIC_CASESexactly twice, and both are prose:The match is assembled from the real imports at the top of the file (lines 10–62), the comment at line 28, and a
from '@objectstack/spec/data'further down the import block. The reference-outside-imports half is satisfied by the comment at line 3233.Why it matters
Today it changes no verdict:
driver-sqlgenuinely coversFILTER_LOGIC_CASESvia a real suite, so the cell is green for the right reason as well as the wrong one. The defect is that the wrong reason is sufficient on its own:consumes()returns the first matching file in walk order, so the file the gate names as the evidence for a cell can be a source file that never runs the case-set. TheRECONCILEDmessage quotes that path back to the author.Possible remedies (not settled — this is why it is filed, not fixed)
import { … } from '…'as a unit rather than letting[\s\S]*?roam. Most correct, and the direction that could turn a currently-green cell red if any driver's coverage is only detected by the loose form — that needs measuring before it is chosen.scripts/js-comment-mask.mjsis the repo's one answer to "comment or code", is validated against@typescript-eslint/parserover 5098 files bycheck-comment-mask-corpus.mjs, and the dialect axis in PR fix(devx): give the driver-conformance census a dialect axis #12134 already routes through it for exactly this reason. Cheapest correct step, and it composes with (1)..tsthat is not a test cannot drive it. This is what the dialect axis does for its own scoring. Narrower than (1) and (2), and it would need checking against drivers whose case-set consumption lives in a non-.test.tshelper.Whichever is chosen, the self-test wants a case pinning that a comment-only mention is not coverage — otherwise the fix is unfalsifiable from a green run.
Dedup note
{"message":"GitHub access is not enabled for this session. An org admin must connect the Claude GitHub App for this organization."}— the known per-session outage. An empty search result here is not evidence that no duplicate exists, and none is claimed. The local half was done: no.changeset/entry, doc, or comment inscripts/check-driver-conformance.mjsrecords this behaviour. Worth a re-check at triage.Related: #12014 (the dialect axis, which surfaced this), PR #12134.
Generated by Claude Code