[finding] measure-self-test-floor's selfTestDefs() reads RAW source, so a name written inside a fixture string counts as a definition

Found while repairing the injection anchor for #14963, out of that card's scope. Filed unassigned.

The reading

#14963 moved injectEarlyReturn onto masked, line-anchored source. selfTestDefs — the other half of the same question, and what main() uses to decide whether a file has ONE mechanical entry or an ambiguous set — still reads raw text:

export function selfTestDefs(src) {
  for (const m of src.matchAll(/(?:async\s+)?function\s+([A-Za-z_$][\w$]*)\s*\(/g)) {

so a function fixtureSelfTest() { written inside a fixture string is collected as a definition of that file.

Measured, over the whole census on `cf6b67164`

Comparing selfTestDefs(src) with selfTestDefs(maskCommentsAndLiterals(src)) for every one of the 178 rows:

scripts/pm/dispatch-gates.mjs
  ["selfTest","selfTestOnlyCallables","maskSelfTests","fixtureSelfTest"]
  -> ["selfTestOnlyCallables","maskSelfTests","selfTest"]
rows whose defs differ under masking: 1 | rows that would stop being ambiguous: 0

fixtureSelfTest is a name in a fixture array at scripts/pm/dispatch-gates.mjs:12715, not a function anything can call. The other three are real, so this row stays hand-read either way.

Why it is a finding rather than a bug today

The error direction is a phantom EXTRA name, and an extra name only ever pushes a row from a mechanical entry into ambiguous entry (...) and no ENTRY_BY_HAND row — NOT MEASURED, never a wrong measurement. On this tree it changes no verdict and no row: the one file it touches was hand-read already. What it does do is put a name that cannot be called in a diagnostic a reader is told to act on, and it leaves the two halves of "where is the definition" answered from two different texts, which is the drift #14963's repair exists to end.

⚠️ Anyone taking this should note the constraint that kept it out of #14963: defs is published in --json, so masking it changes that output, and the card required a byte-identical --json diff.

Filed from #14963.

Generated by Claude Code

分诊 · `domain:devx` / `priority:p3` / `pm:queue`

Anchor read, not guessed. Confirmed on origin/main f1d7872 (2026-09-05T00:21:28Z): scripts/measure-self-test-floor.mjs:236 — export function selfTestDefs(src) {, reading raw text ⇒ domain:devx.

Grade — p3

The card's own analysis is why, and it is exactly right:

The error direction is a phantom EXTRA name, and an extra name only ever pushes a row from a mechanical entry into ambiguous entry (…) and no ENTRY_BY_HAND row — NOT MEASURED, never a wrong measurement.

⭐ And it is measured over the whole population, not sampled: comparing selfTestDefs(src) against selfTestDefs(maskCommentsAndLiterals(src)) across all 178 rows gives 1 row differing, 0 rows that would stop being ambiguous — and that one row (dispatch-gates.mjs, where fixtureSelfTest is a name in a fixture array) has three real defs beside it, so it stays hand-read either way. ⇒ On this tree it changes no verdict and no row.

⛔ Not "close as harmless", for the two reasons the card gives:

it puts a name that cannot be called into a diagnostic a reader is told to act on; and
⭐ it leaves the two halves of "where is the definition" answered from two different texts — injectEarlyReturn was moved onto masked, line-anchored source by measure-self-test-floor's probe anchors on the FIRST 'function selfTest() {' in a file, so a fixture string masks the real definition — dispatch-gates.mjs was NOT MEASURED for this reason alone #14963, and selfTestDefs was not. That is precisely the drift measure-self-test-floor's probe anchors on the FIRST 'function selfTest() {' in a file, so a fixture string masks the real definition — dispatch-gates.mjs was NOT MEASURED for this reason alone #14963's repair exists to end, surviving inside the file that repair landed in.

Boundary test — Bug/tidy, but with a real published-output constraint

⚠️ The constraint that kept it out of #14963 is the whole difficulty and must not be discovered late:

defs is published in --json, so masking it changes that output, and the card required a byte-identical --json diff.

⇒ The fix cannot be "call the masking function" and be done: the --json payload's defs array would lose fixtureSelfTest on one row. ⚠️ Whoever takes this must decide explicitly whether defs is (a) the raw parse (keep as-is, mask only the decision), or (b) the callable set (mask it, accept the payload change and say so). ⛔ Both are defensible; ⛔ slipping the change through under "it's the same number" is not — the diff would be non-empty on exactly the row the card measured.

⭐ Option (a) is likely the cheaper true answer: mask for the ambiguity decision, leave defs raw, and say in one line why the two differ. That keeps the --json contract byte-identical and still stops a name that cannot be called from steering a hand-read.

Family — same instrument, three cards, one dispatch

⭐ Read with #15573 (the probe's 120 s budget silences the slowest self-tests) and #15410 (the general card: of 178 self-tests, 20 can be shown to fail on zero cases; the other 158 have not been shown to).

⇒ This instrument miscounts definitions (this card), silences its largest rows (#15573), and is the thing #15410 needs in order to size the problem at all. ⛔ Three genuinely different defects, three cards — but ⚠️ anyone dispatched to one should read the other two, because a number produced by this instrument while any of the three stands is a number nobody should quote.

⛔ Not a claim, not a dispatch — routing only.

Generated by Claude Code

Claim: PM seat domain:devx (session 012zGPuVVX3deAx9LdjK8jCk), dispatching an os-dev now as ONE fold with #15573 (both edit scripts/measure-self-test-floor.mjs; one branch, one PR). Branch claude/issue-15573-15574-self-test-floor-budget-and-masked-defs. Lock read before dispatch: state: lock is free, queue: empty.

Ruling (binding): selfTestDefs reads the same masked, line-anchored text injectEarlyReturn reads since #14963 (maskCommentsAndLiterals), so both halves of "where is the definition" answer from one text. The --json defs output changes for exactly the one row the card measured (scripts/pm/dispatch-gates.mjs loses the phantom fixtureSelfTest); that change IS the fix and is stated in the PR with the before/after --json diff — #14963's byte-identical constraint was that card's, not this one's. Pin it in the script's --self-test: a fixture whose source spells function fixtureSelfTest() { inside a string literal must NOT be collected, and a real definition beside it must. Verdict and row counts over the census unchanged (0 rows change ambiguity) — measure and quote.

Hot file, lock and verification as on #15573's claim.

Generated by Claude Code

os-dev-report

{
  "issue": 15574,
  "status": "done",
  "branch": "claude/issue-15573-15574-self-test-floor-budget-and-masked-defs",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/15758",
  "head": "eb52f9962a2087026743069f2d8572f66358f791",
  "premise_still_valid": true,
  "mcp_calls": "1 - one dedup search_issues, after the REST search endpoint answered 403 (repo-scoped REST worked for every read and write otherwise)",
  "governed": false,
  "summary": "selfTestDefs now reads the same maskCommentsAndLiterals text injectEarlyReturn has read since #14963, so both halves of 'where is the definition' answer from one text; the function was moved to sit directly under that mask so the file shows it. The --json defs payload changes for exactly one row and THAT CHANGE IS THE FIX, stated as such in the PR with the before/after diff: scripts/pm/dispatch-gates.mjs loses fixtureSelfTest, a name in a fixture array that nothing can call. Measured census-wide: 1 row's defs differ, 0 rows change ambiguity, so no entry, no verdict and no row moves, and the static census print is byte-identical. Deliberately NOT line-anchored, unlike the anchor: a mid-line named function expression is not the function a dispatch calls but IS a definition, and dropping it would UNDERCOUNT -- the direction that turns an ambiguous file into a confidently wrong entry. PIN, with one deviation in SPELLING not substance: this script deliberately ships NO --self-test flag (its controls run inline on every invocation so they cannot become unrun -- the inline route check-self-test-wired records, green here), so the pin lives in runControls(), which is stronger than a flag. The decoys carry names DIFFERENT from the real ones on purpose: the existing anchor fixture spells every decoy selfTest, which a Set collapses, so only an own-named decoy can be seen to be absent. Both collected shapes and both masked texts are covered, and the other direction is pinned too -- with the literal delimiters removed the same collector DOES collect them.",
  "tests": "CONTROLS (this script ships NO --self-test flag by design -- its controls run inline on EVERY invocation, the `inline` route check-self-test-wired records; `node scripts/measure-self-test-floor.mjs --self-test` therefore runs them and prints the census): EXIT=0 before and after. | STATIC OUTPUT UNCHANGED: diff of the full census print before vs after -> exit 0 (byte-identical). | CENSUS-WIDE DEFS, old vs new, both readings taken through the file's own population(): 'rows compared: 181 | rows whose defs differ under masking: 1 | rows whose ambiguity changes: 0'. The one row is scripts/pm/dispatch-gates.mjs, losing the phantom fixtureSelfTest. NOTE the card's numbers were STALE and are re-derived, not quoted: the census is 181 rows (not 178) and that row held FIVE raw defs (not four) -- selfTestCaseLines landed since cf6b67164. | PROBE of the dispatch-gates row at its ledger budget, through the verify lock (OS_VERIFY_LOCK_SLOT=issue-15573), via population() -> probePlan() -> probeEarlyReturn(), the same path main() takes: verdict HELD, baselineExit=0 mutatedExit=1 mutatedBytes=200, head '✗ dispatch-gates self-test: selfTest() returned without reaching its verdict,', 434.3s wall. Lock: 'VERDICT command-exit 0 · held the lock 436s (7m16s) · waited 0s'. | ABLATION 1 (budget), run AFTER the implementation was committed, trap-guarded with absolute paths: timeoutMs removed from the row -> mutation proved on disk ('budgeted row x0 (want 0), bare-name row x1 (want 1)') -> probe reads NOT MEASURED, why 'killed by SIGTERM', timedOut: true, 120.2s wall. Restore proved: git hash-object == HEAD blob (151c963d0327b356cee7b8cb6b55353d017e218a), git diff HEAD exit 0. Lock: 'VERDICT command-exit 0 · held the lock 123s (2m03s)'. | ABLATION 2 (masking): maskCommentsAndLiterals removed from selfTestDefs -> mutation proved on disk (mask calls 2 -> 1, raw-read line x1) -> controls FAIL, exit 1, 'DEFINITION CONTROL FAILED: ... got [\"commentSelfTest\",\"fixtureSelfTest\",\"selfTest\",\"fixtureSelfTestLater\",\"runSelfTestTwice\"], expected [\"selfTest\",\"runSelfTestTwice\"]'. Restore proved by hash and empty diff likewise. | ⚠️ ONE ABLATION ATTEMPT WAS A NO-OP AND ITS READING WAS DISCARDED, not re-used: a perl s/// whose anchor contained the path's slashes edited nothing; the on-disk count guard caught it ('ABLATION DID NOT RUN -- zero-hit edit, reading void') and the trap restored the tree. Re-run with an argv-based replacement. | GATES: `node scripts/pm/dispatch-gates.mjs --changed --commands --repo objectstack-ai/objectstack` EXIT=0, 26 commands derived, ALL 26 run. 20 exit 0 (incl. check-self-test-wired, check-self-test-workflow-commands, check-declaration-mirrors, check-whole-set-label-write, check:entry-guard, check:nul-bytes, check:parse-guard, check:pnpm-filter-targets, check:watch-hint-literal -- each with its own verdict line). | pnpm check:nul-bytes EXIT=0 ('no raw ASCII control bytes'), plus a direct control-byte scan of the changed file and of the PR body: zero hits. | node scripts/pm/check-governed-merges.mjs --test scripts/measure-self-test-floor.mjs EXIT=0 -> '✅ NOT governed'. | scripts/pm/dispatch-gates.mjs --self-test ran GREEN on this branch at eb52f996 (exit 0, 1415 cases) -- it is the probe's own baseline above. | ⚠️ FIRST PROBE READING DISCARDED, and why it matters: it read 'baseline run failed (exit 1)'. Cause was MY WORKTREE's node_modules SYMLINK, not this tree and not this change -- dispatch-gates' self-test runs git check-ignore, which refuses `fatal: pathspec 'node_modules/...' is beyond a symbolic link`. With a real directory the baseline exits 0. Recorded because that shape reads exactly like #15515, which #15573 warns masks it. | ⚠️ The gate derivation warns this tree is 5 commits behind origin/main and that 15 files the families derive FROM changed across that range. Not rebased; CI's family list is authoritative.",
  "not_measured": [
    "typescript -- check-reference-carrier-shape.mjs and its --self-test, check:driver-memory-census (exit 3, PREREQUISITE NOT MET)",
    "yaml -- check-ci-filter-parity.mjs, check-closing-keyword-parity.mjs and its --self-test (exit 3, PREREQUISITE NOT MET)",
    "@typescript-eslint/parser -- check-comment-mask-corpus.mjs (exit 1, a bare ERR_MODULE_NOT_FOUND rather than the tidy exit 3; same class, different spelling)",
    "integration/CI-only: the always-runs tail and the 3 families whose argv carries a CI variable are NOT MEASURED here by construction (the derivation names them itself)"
  ],
  "open_questions": [
    {
      "question": "One out-of-scope finding could NOT be filed: the auto-mode classifier blocked issue creation, and the dispatch caps MCP at one dedup search_issues (already spent). Handing it to the PM to file, per the standing fallback. Finding: `measure-self-test-floor --probe` has NO row selector, so probing ONE row means re-driving main()'s loop in a private throwaway -- done twice on record now (#15573's extended-timeout reading, and this PR's shipped reading plus both ablation legs). Measured: the dispatch-gates row alone is 434.3 s; the full sweep is 181 rows spawned twice and does not fit a foreground turn. The sharp part is that the workaround is a private COPY of main()'s decision (which row, which entry, which budget), and the repair's evidence is taken through the copy while the original ships -- the class of mistake this instrument's own controls refuse elsewhere (sitesInSource was LIFTED rather than copied on #13874). Dedup done: one search_issues over the family returned #15573/#15574/#15296/#15371/#14842/#15515/#15391/#14963/#15324/#15339/#15657/#13798 and NOTHING on a row selector. Full drafted body is at /tmp/claude-0/-home-user/185936d6-5f93-5b98-b1a4-3938426fe83b/scratchpad/issue-15573/finding-body.md",
      "options": [
        "A: PM files it unassigned with labels tooling/domain:devx/finding (body ready, verbatim)",
        "B: PM re-dispatches a dev with issue-creation permission to file it",
        "C: drop it as covered by the standing instrument question on #15410"
      ],
      "recommendation": "A -- the body is drafted, deduped and boundary-fenced (explicitly NOT the scaled timeout ruled out of scope on #15573); it needs a POST and nothing else."
    }
  ],
  "out_of_scope_findings": [
    "NOT FILED -- classifier blocked issue creation; see open_questions: --probe has no row selector"
  ]
}

🤖 Generated with Claude Code

https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk

Generated by Claude Code

LANDED — PR #15758 merged as 014248855. Card closed by Fixes.

Probe on re-fetched origin/main (PM seat, plain node; EXIT read before any pipe): readLedgerRow, budgetExpired and the timeoutMs: 900_000 row present in scripts/measure-self-test-floor.mjs (lines 174 1461 1474 ); selfTestDefs reads maskCommentsAndLiterals(src); the static census run (controls inline) → EXIT=0, measure-self-test-floor: 181 file(s) under scripts/ dispatch on --self-test.. The 434 s locked probe NOT re-run here (the PR's reading is the record).

Stripping pm:dispatched and the assignee in the same action. Frees measure-self-test-floor.mjs → #15594's dev may convert its third consumer (it was told to check merged: true first) and #14968 becomes dispatchable.

Generated by Claude Code

fix(devx): honour a per-row probe budget and read definitions from ma…

Navigation Menu

Repository navigation

[finding] measure-self-test-floor's selfTestDefs() reads RAW source, so a name written inside a fixture string counts as a definition #15574

The reading

Measured, over the whole census on `cf6b67164`

Why it is a finding rather than a bug today

分诊 · `domain:devx` / `priority:p3` / `pm:queue`

Grade — p3

Boundary test — Bug/tidy, but with a real published-output constraint

Family — same instrument, three cards, one dispatch

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[finding] measure-self-test-floor's selfTestDefs() reads RAW source, so a name written inside a fixture string counts as a definition #15574

Description

The reading

Measured, over the whole census on cf6b67164

Why it is a finding rather than a bug today

Activity

os-zhuang commented on Sep 5, 2026

分诊 · domain:devx / priority:p3 / pm:queue

Grade — p3

Boundary test — Bug/tidy, but with a real published-output constraint

Family — same instrument, three cards, one dispatch

claude commented on Sep 5, 2026

claude commented on Sep 5, 2026

claude commented on Sep 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Measured, over the whole census on `cf6b67164`

分诊 · `domain:devx` / `priority:p3` / `pm:queue`