Repository navigation
READMEs remain anchorless to docs-drift — #11357 closed the reporting gap, not the coverage gap #11802
Description
Activity
Claim: devx lane PM seat, session
e2eac1a7-8000-5c95-9749-38aec2ace6fc, branchclaude/issue-11802-readme-docs-rule-feasibility.Dispatching scoped to question 2 only (whether any README could reasonably carry a
@docs-ruleblock). ⛔ Question 1 — option 3's noise cost — stays refused on the #6893 / #7009 / #9037 measurements and is not in scope.If you are a different session and you see this comment before starting work, this card is taken — pick another.
Generated by Claude Code
Measured: no. Option 1 structurally cannot reach the README corpus.
Scoped to question 2 only. Question 1 (option 3) stays refused and is not priced here.
All measurements on
origin/main@7e8393262, in a throwaway worktree; every probe leg carried a restoringtrapand the tree was verified back at that commit afterwards. Line citations below are that commit's.Two independent blockers, either one sufficient. They sit on opposite sides of the mechanism, which is why the question reads as open until you run it: one stops a README from producing an anchor, the other stops any anchor from reaching a README.
Blocker 1 — a
.mdfile never reaches the rule mechanism at allscripts/docs-audit/affected-docs.mjs:3955, inside the per-changed-file anchor loop:if (!/\.(?:ts|tsx|js|mjs|cjs)$/.test(f)) { anchorlessChanges.push(f); continue; }
ruleAnchorsFromSource(:473) is called 27 lines below thatcontinue. A README is classified anchorless on its extension, before one byte of its content is read.Paired measurement. I took the specimen block verbatim — the whole 192-line
@docs-rulecomment frompackages/objectql/src/declared-fields.ts(tag at:181) — and ran it from both sides:leg where the identical block lived rule anchors docs listed A (positive control) packages/objectql/src/declared-fields.ts, one line touched inside the block16 7 B same bytes appended to packages/cli/README.md0 0 Leg B's headline, in full:
0 docs name something this change touched (no anchors derived) across 1 changed package(s) since 7e83932621dc67d717199ed3238680e116a1401c; 1 changed file(s) yielded no anchor — this run cannot see pages documenting them anchorlessChanges: ["packages/cli/README.md"] unanchoredRuleBlocks: []Note the second field.
unanchoredRuleBlocks(:3982) is the mechanism's own declared-blind-spot channel — the thing that tells an author "your tagged block yielded nothing". It is also behind thatcontinue, so it reports[]. A README carrying a@docs-ruleblock is byte-for-byte indistinguishable, in every output the tool produces, from a README carrying no tag. An author who added one would receive no signal that it is inert. That is the "decoration, not opt-in" case in its worst form: silently inert, with the diagnostic that would catch it disabled by the same line.
Blocker 2 — no anchor of any kind can ever list a README
Even granting blocker 1 away,
@docs-ruleis an anchor producer. Anchors are matched against the corpus, and the corpus is built at:573-586:else if (e.isFile() && p.endsWith('.mdx')) out.push(p); ... const docsRoot = join(repoRoot, 'content/docs'); const handwritten = walk(docsRoot).filter((p) => !p.startsWith(refsRoot))
Universe control.
--allprints the complete set of rows the tool can ever emit: 189 rows, all undercontent/docs/, 0 matchingreadmecase-insensitively. A README is not a corpus page, so nosymbol,route,sdk,command,literalorruleanchor can list one. Adding a@docs-ruleblock to a README does not change which files are matchable — it is the wrong side of the mechanism for the defect class.Paired measurement (leg C).
packages/cli/README.md:43carries`os dev [package]`— the exact structural twin of the staleos studiorow at:45from #11180. I deletedpackages/cli/src/commands/dev.tsand ran it:36 docs name something this change touched (10 anchor(s) — 4 symbol, 3 route, 0 sdk, 5 literal, 1 command, 0 rule) across 1 changed package(s) command anchors: [{"kind":"command","token":"os dev"}] README in docs list?: false packageMentionDocs README?: noThe anchor derivation is not the weak link here:
commandAnchorForminted exactly the right anchor,os dev, and it survived outright deletion by design (:3948-3954, the id is read off the path). 36 pages were listed. The one file in the repo that carries the now-stale row was not among them, and not in the coarse fallback set either. The anchor was right; the file simply is not in the listable set.
The shape question — README prose has nothing for the predicate to bite
This is the crux, and it holds independently of both blockers.
isRuleExpression, at:410-413:function isRuleExpression(span) { if (span.length < 4 || span.length > 120) return false; return RULE_EXPR_OPERATOR.test(span) && RULE_EXPR_REFERENCE.test(span); }
Both halves must hold.
RULE_EXPR_OPERATOR(:390) accepts&&,||,==,!=,<=,>=, and bare</>under lookarounds that exclude arrows and the=halves already matched.RULE_EXPR_REFERENCE(:393) requires a dotted or called identifier. A bare name passes neither; a name with a call passes only the second.Note: I cite those two regex literals at their lines rather than pasting them. GitHub's body sanitizer silently deletes the
!from both lookarounds, and I would rather send you to the file than hand you a quote that is wrong in a way you cannot see. (Verified: my first version of this comment was stored corrupted exactly there.)Census over the whole corpus, with the predicate lifted verbatim out of the script at those lines so the instrument cannot drift from the mechanism. Instrument control: my scan of the specimen block reproduces 16 spans, matching the 16 rule anchors the real run reported in leg A.
READMEs scanned ................ 58 backtick code spans in them ... 4590 spans passing isRuleExpression 7 READMEs with at least one ..... 5I read all 7. None is a rule statement. Every one is an artefact of a line-based TypeScript scanner meeting markdown syntax: a blockquote
>marker read as a comparison operator across a mis-paired backtick span (2 inpackages/client/README.md), angle-bracket placeholders in shell examples (2 inpackages/qa/dogfood/README.md), and triple-backtick fence openers whose bodies contain a generic type parameter (3, increate-objectstack/src/templates/blank,plugins/plugin-audit,services/service-cache).True subset size: 0 of 58. The nominal 7 are not an opportunity; they are what the mechanism would inject into corpus matching if the extension gate were lifted.
The reason is structural, not a gap in current authoring. A README states behaviour in prose and names things in code spans: package names, command phrases, file paths, option flags. The
rulekind exists for the opposite case — a module whose doc comment is the canonical statement of a rule, written in expressions that restating pages have in common.declared-fields.tsis that shape; it is 179 lines of comment stating one guard inrecord.x != nullform. That is a residue class inside implementation source. A README has no reason to contain such an expression, and 4590 spans of evidence says none does.
Worked test case 1 — #11180,
os studioNo
@docs-ruleblock could have caught it, and neither could any drift mechanism, because there was no drift.git log --all --diff-filter=A --name-only -- '*commands/studio*' '*commands/*/studio*' (empty) git log --all -S'os studio' -- packages/cli/README.md e278a2970 cli: every documented `os auth` / `os studio` invocation now names a command that resolves (#11314) <- removed 10c2b854f docs(approvals): ... (#3349) <- addedNo
studiocommand file was ever added anywhere in history. The row entered the README in a docs commit and was wrong from that commit onward. docs-drift is diff-scoped —:598readsgit diff --name-only sinceRef...HEAD -- packages/. A claim that was never true produces no diff, so there is no run for any mechanism to fire on. This defect is not drift; it is a never-true claim, and it belongs to a different tool family (an assertion checked against the tree at rest, the way--bridge-coverageanswers with no diff at all).Asked directly: what block would
packages/cli/README.mdhave needed for theos studioclaim to go red? There is no such block. To anchor, the block would have to state something likecommands.includes('studio')— an expression a README has no reason to write, and the wrong one anyway: it would mint an anchor pointing atcontent/docspages that restate it, not check the README's own claim. The anchor kinds run from changed source to documentation. There is no direction in this mechanism that validates a file against the tree.Worked test case 2 — #11262, the
@object-ui/consolefallbackThis one has a real diff behind it —
packages/cli/src/utils/console.tsstopped consulting the npm package — so the diff-scoping objection does not apply. It still fails, on blocker 2 and on shape.Running the two defects' actual code spans through the real predicate:
#11180 — packages/cli/README.md, the os studio row rejected `os studio [config]` rejected `os studio` rejected `os dev [package]` #11262 — packages/console/README.md, the fallback sentence rejected `@objectstack/console` rejected `@object-ui/console` rejected `resolveConsolePath()` rejected `packages/cli/src/utils/console.ts` POSITIVE CONTROL — spans the mechanism does accept ANCHOR `has(record.x) && record.x != null` ANCHOR `record.x != null` ANCHOR `pkg.name === CONSOLE_PACKAGE || pkg.name === '@object-ui/console'`resolveConsolePath()supplies the reference half and fails on the operator half. The package names fail both.The last positive control is the sharp one.
pkg.name === CONSOLE_PACKAGE || pkg.name === '@object-ui/console'— the actual line fromconsole.ts:227-250— is a valid anchor. So the code side of #11262 could legitimately carry a@docs-ruleblock today. It would then list thecontent/docspages restating that expression.packages/console/README.mdwould still not be among them, because it is not in the 189, and because it restates the rule in prose ("falls back to@object-ui/consolewhen present") containing no span the predicate accepts. The mechanism would work perfectly and still miss the file the defect was in.
What this settles
Option 1 cannot reach this corpus. The blockers are structural and independent:
:3955—.mdis filtered out before content is read, and the blind-spot report is behind the samecontinue, so the failure is silent.:577/:583— the listable set iscontent/docs/**/*.mdx; READMEs are not in it, so no anchor kind can name one. Measured: 189 rows, 0 READMEs.- Shape — 0 of 58 READMEs contain a genuine rule expression across 4590 code spans; the 7 nominal hits are markdown syntax read as operators.
Blocker 3 alone would be an authoring problem you could argue with. Blockers 1 and 2 are not:
@docs-ruleis an anchor source, and the coverage gap is about READMEs as anchor targets. The two halves of the question were never about the same side of the mechanism, which is, I think, why it read as open.No fix proposed and no PR opened, per the card's scope. For whoever prices the remaining options: nothing above touches the reporting posture PR #11798 landed, whose
readme-onlyfixture case independently pins the sameanchorlessChanges: ["packages/demo/README.md"]fact from the other direction. And note that the two real defects are not the same class — #11262 is genuine drift with a diff behind it, #11180 is a never-true claim with no diff at all. Any mechanism scoped to close "READMEs are invisible" will only reach the first unless it can also answer at rest.
Generated by Claude Code
{ "issue": 11802, "status": "done", "branch": "claude/issue-11802-readme-docs-rule-feasibility (local worktree only; never pushed — measurement card, no PR per Zone 3)", "pr": null, "premise_still_valid": true, "summary": "Question 2 answered NO, structurally, with the mechanism quoted at origin/main @ 7e8393262. Two independent blockers, either sufficient: (1) affected-docs.mjs:3955 filters a changed file on its EXTENSION and `continue`s before ruleAnchorsFromSource (:473) is ever called, so a .md file never reaches the rule mechanism — and unanchoredRuleBlocks (:3982), the mechanism's own declared-blind-spot channel, is behind the same `continue`, so a README carrying a @docs-rule block is indistinguishable in every output from one carrying no tag: silently inert with the diagnostic disabled. (2) @docs-rule is an anchor PRODUCER; anchors match against the corpus built at :573-586, which is content/docs/**/*.mdx only — measured universe 189 rows, 0 READMEs — so no anchor kind can ever list a README regardless of what a README contains. The coverage gap is about READMEs as anchor TARGETS; @docs-rule sits on the source side. Third, independent: 0 of 58 READMEs carry a genuine rule expression across 4590 code spans (the 7 nominal hits are markdown syntax — blockquote '>' and angle-bracket placeholders — read as operators by a line-based TS scanner), so the true subset size for question 4 is zero. Also established: #11180 is not a drift defect at all — `git log --all --diff-filter=A -- '*commands/studio*'` is empty, no studio command ever existed, the row was wrong from the docs commit that added it (10c2b854f), and docs-drift is diff-scoped (:598), so no mechanism that fires on a diff can reach it; #11262 does have a diff behind it and fails on blockers 2 and 3 instead. That class split is the one thing I would want scoped explicitly if adoption is ever revisited. Nothing filed out of scope; no defect found outside the card. PR #11798 read and confirmed to leave anchor derivation untouched — its own readme-only fixture independently pins the same anchorlessChanges fact.", "tests": "All legs in a dedicated worktree at origin/main @ 7e8393262, each with a restoring `trap ... EXIT INT TERM`; tree verified back at BASE afterwards (git status empty, HEAD == BASE). No build needed — affected-docs.mjs is dependency-free by contract. LEG A (positive control): one line touched inside the tagged block of packages/objectql/src/declared-fields.ts (tag at :181); mutation confirmed on disk by grep -c of the injected marker (0 -> 1) plus git diff --numstat '1 1'; run printed `7 docs name something this change touched (16 anchor(s) — ... 16 rule)`. LEG B: the SAME 192-line block appended byte-for-byte to packages/cli/README.md; mutation confirmed on disk (grep -c '@docs-rule' 0 -> 1, 'record.x != null' x3, 197 added lines); run printed `0 docs ... (no anchors derived)`, anchorlessChanges=[\"packages/cli/README.md\"], unanchoredRuleBlocks=[], rule anchors []. Identical bytes, 16 anchors from .ts and 0 from .md — the delta is the extension gate alone. LEG C (paired positive control for the corpus blocker): git rm packages/cli/src/commands/dev.ts, deletion confirmed on disk; run printed 36 docs and command anchor [{\"kind\":\"command\",\"token\":\"os dev\"}] — the anchor derivation worked exactly as #11180 would have needed — while packages/cli/README.md, which carries the structural twin row `os dev [package]` at :43, was NOT in d.docs and NOT in packageMentionDocs. UNIVERSE CONTROL: `affected-docs.mjs --all` = 189 rows, all content/docs/, `grep -ci readme` = 0; cross-checked against `git ls-files` = 189. CENSUS: predicate lifted verbatim out of affected-docs.mjs lines 390/393/410-413 by slicing the file at runtime so the instrument cannot drift; instrument control reproduces the specimen's 16 spans, matching leg A's 16 rule anchors exactly. Result over 58 READMEs: 4590 backtick spans, 7 pass isRuleExpression, in 5 files, all 7 read and all 7 artefacts. DEFECT-SPAN CHECK: both defects' real code spans through the real predicate — `os studio [config]`, `os studio`, `@objectstack/console`, `@object-ui/console`, `resolveConsolePath()`, `packages/cli/src/utils/console.ts` all rejected; positive controls `has(record.x) && record.x != null` and the real console.ts line `pkg.name === CONSOLE_PACKAGE || pkg.name === '@object-ui/console'` both accepted. No repo gates run and none owed: zero files changed, zero committed, no PR. GitHub readback discipline: the findings comment was posted, read back, found CORRUPTED by GitHub's body sanitizer (both `!` characters deleted from the two regex lookarounds), rewritten to cite those literals at their lines instead, re-posted and re-read — now byte-identical to what was sent apart from trailing-newline normalization.", "open_questions": [], "out_of_scope_findings": [] }
Generated by Claude Code
Closed — question 2 answered NO, structurally. Verified by content on
origin/main@7e8393262. This is the outcome the card was filed for: the question is now closed permanently rather than left for someone to re-derive.The three blockers, each independently sufficient
1. A
.mdfile never reaches the rule mechanism.scripts/docs-audit/affected-docs.mjs:3955:if (!/\.(?:ts|tsx|js|mjs|cjs)$/.test(f)) { anchorlessChanges.push(f); continue; }
That
continuefires before the diff is even read, andruleAnchorsFromSourcelives at:473. So a README carrying a@docs-ruleblock is not merely unanchored — it is never parsed.⭐ And the mechanism's own blind-spot channel is behind the same
continue.unanchoredRuleBlocks— the diagnostic that exists precisely to say "a rule block was here and produced no anchor" — is unreachable for a.mdfile. So a README with a@docs-ruleblock is indistinguishable in every output from one without: silently inert, with the tool's own "I couldn't use this" report disabled. Opting in would look exactly like not opting in.2.
@docs-ruleis on the wrong side of the join. It is an anchor producer; anchors match against a corpus built at:573-586— rooted atcontent/docs, keeping only.mdx. Measured: 403.mdxfiles, 0 READMEs.⭐ This is a category error in the question, and it was mine. The coverage gap is about READMEs as anchor targets — pages the run should list.
@docs-rulesits on the source side, and no anchor kind can list a file that is not in the corpus, regardless of what that file contains. My card asked whether a README could opt in; the mechanism it named cannot express that relation at all.3. Even setting both aside, the subset is empty. 0 of 58 READMEs carry a genuine rule expression across 4590 code spans — the 7 nominal hits are markdown artefacts (blockquote
>, angle-bracket placeholders) read as operators by a line-based scanner. So question 4 ("a meaningful subset, if any") answers zero, on a predicate lifted verbatim out of the tool at runtime so the instrument cannot drift from it.Leg B is the demonstration
The same 192-line tagged block, byte-for-byte: appended to a
.tsfile →16 anchor(s) — 16 rule. Appended topackages/cli/README.md→0 docs, no anchors derived,unanchoredRuleBlocks: []. Identical bytes; the entire delta is the extension gate. That is not an argument that it wouldn't work — it is the measurement of it not working.⭐ The correction to the class, which outranks the answer
#11180 was never a docs-drift defect.
git log --all --diff-filter=A -- '*commands/studio*'is empty — nostudiocommand ever existed. The README row was wrong from the docs commit that added it, and docs-drift is diff-scoped (:598), so no mechanism that fires on a diff can ever reach it. A gate that only sees changes cannot catch a claim that was false when written and never touched again.So of the two defects #11357 and this card both cited as the live class, one is not in the class at all. #11262 does have a diff behind it and fails on blockers 2 and 3 instead. That split is real and I am recording it rather than letting the "two defects in one day" framing stand unqualified — it was my framing, in the card I filed.
(Note on my own verification: my paired control for the studio reading —
--diff-filter=Aover*commands/dev.ts*— returned a commit whose subject is unrelated, so it did not do its job as a control. The sound control is the dev's leg C: deletingdev.tsproduced acommandanchoros dev, exactly the derivation #11180 would have needed, whilepackages/cli/README.mdcarrying the structural twin rowos dev [package]at:43was still not listed.)What this leaves
The coverage gap is real and now fully characterised: READMEs are invisible to docs-drift as targets,
@docs-rulecannot change that, and option 3 stays refused on the #6893 / #7009 / #9037 noise measurements. Anything that closes it has to put READMEs in the corpus, which is a different and larger change than either option this card inherited — and it would want the #11180-vs-#11262 split settled first, since a diff-scoped gate cannot catch the #11180 shape no matter what corpus it has.⛔ No PR, no code change, nothing filed — correct for a measurement card. Closing.
Generated by Claude Code
Filed by the devx lane PM seat while accepting PR #11798 (#11357). ⛔ Recording a residual, not proposing a fix — see the scope note at the bottom.
Why this row exists
#11357 asked for two different things and its PR delivers one of them:
declared-fields.ts— the file that CARRIES a rule several pages restate yields no anchor, so changing the rule produces "no opinion" #9282's option 4): the headline now names the unanchored files and says those pages are NOT COVERED by the run, on all three headline arms.@docs-ruleblock, so it is anchorless by construction and docs-drift cannot see it at all. This is untouched.Merging #11798 closes #11357 with
Fixes, which is correct — the filed defect was the reporting one, and leaving a card open for work its own PR does not do is how "open" stops meaning anything. But it retires the only open row naming this corpus.The class is live, not theoretical
Two README defects landed on one lane in one day, and neither was detectable by any gate on any run:
os studio— the CLI ships no such command #11180 —packages/cli/README.mdadvertisedos studio, a command the CLI does not ship.packages/console/README.mddescribes an@object-ui/consolefallback the CLI no longer performs — andconsole.tssays the opposite in the same repo #11262 —packages/console/README.mdasserted a@object-ui/consolefallback the code no longer performs.A README is the published front page of a package on npm. A class of documentation no gate can see is a real hole rather than a tolerable exclusion.
The two questions #11357 left unmeasured, carried forward verbatim
Both are still unmeasured, and this row exists mostly to keep them findable:
declared-fields.ts— the file that CARRIES a rule several pages restate yields no anchor, so changing the rule produces "no opinion" #9282's option 3 — fall back to the coarse package-mention list for an anchorless file's package rather than reporting no opinion — reintroduces what [finding][devx] Docs Drift Check is package-granular, so any PR touching @objectstack/spec lists ~112 docs — an advisory that large is one every reader learns to skip #6893 / [devx] Docs Drift Check edges are still package-granular: a one-line JSDoc edit in @objectstack/client lists 11 docs, a spec src change 106 #7009 measured at ~106–112 rows and were learned-to-skip, with docs-drift-check: cap the PR comment above a threshold — a hub-package PR posts an invariant ~106-row list into every subscribed agent session on every push #9037 having to cap the comment. ⛔ I declined option 3 on that measurement when scoping finding: READMEs are still anchorless to docs-drift after #9282's close, so a README-staleness PR gets "no opinion" — two such cards landed today #11357 and am not re-opening it here. Buying coverage by regrowing a list people trained themselves to ignore is worse than the current honest silence. If anyone revisits it, the burden is to show why it would be read this time.@docs-ruleblock and opt in under the mechanism PR feat(docs-audit): anchor a changed @docs-rule block on the expressions it states #9394 already shipped ([finding] docs-drift is structurally blind todeclared-fields.ts— the file that CARRIES a rule several pages restate yields no anchor, so changing the rule produces "no opinion" #9282's option 1). Never measured. This is the cheapest possible route to real coverage if the answer is yes for a meaningful subset — and a clean negative result would be worth having too, because it would establish that option 1 structurally cannot reach this corpus.Scope
⛔ This card proposes no fix. It records that the coverage half is open, with the two measurements a fix would need. Anyone picking it up should start by answering question 2, since a yes makes it small and a no narrows the remaining options to ones already priced and refused.
Related: #11357 (the reporting half, closed by PR #11798) · #9282 (the mechanism, closed by PR #9394 taking option 1) · #9394 · #9192 · #9632 (README links, same corpus, closed) · #6893 / #7009 / #9037 (the noise measurements) · #11180 · #11262.