Repository navigation
finding: no gate reads a doc snippet's schema KEYS against the spec — #4823's deferred "second dimension", closed unbuilt #5138
Description
Activity
PM triage — split ruling: shape 2 →
pm:queue; shape 1 stays a finding, unruled. PM sessionsession_01NYgmGheCzM6NrHZN436Cxf.The card asks for a maintainer call on scope and cost. Part of it is not one, and I am ruling that part; the rest genuinely is, and I am leaving it alone rather than deciding it quietly.
The pattern this belongs to
Third instance recorded tonight, across two repos, of one shape — a literal that is contract on one surface and unguarded on another:
card literal pinned on unguarded on objectstack#9491 required check-run names workflow files AGENTS.md,.claude/skills/**#5138 spec schema keys code ( ReportSchema, strict)doc snippets #5151 shell syntax nothing every fenced block Worth naming because the framing changes what a fix should optimise for. In all three the scan set is not the gap — the files are read. The gap is the assertion. #9491 measured this precisely with a positive control (the id-lint gate goes red on the same line for a different reason), and this card measures the same thing from the other side: three gates green on prose containing four separate falsehoods.
The line in this card I want carried into whatever gets built:
The gate looked at the one key that was correct.
A gate that checks the one thing that is right, while everything around it is wrong, is worse than no gate — it converts "unverified" into a green. That is the failure mode to design against, not snippet coverage per se.
Ruled: shape 2 →
pm:queuePromote the snippet-extraction +
tsc --strict-against-built-distharness intoscripts/. Reasoning by the four standing dimensions, and it is dimension ④ that decides it:- ① Long-term coherence — three private copies of one harness is duplication with a known direction of travel.
- ② Measured pull — six instances named on this card (schema-catalog: all 9
plugin-dashboardexamples are refused by the spec'sDashboardSchema— the AI few-shot corpus teaches the pre-ADR-0021 widget shape #4600, plugin-report README 教三个不存在的导出:ReportBuilder组件、registerDrillHandler、ScheduleConfig组件 #5016, plugin-report README's export snippets call every real export with the wrong signature —exportAsCSV(data, 'file.csv')passes a filename where the data argument goes #5048, plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047, plugin-gantt README 教的整片 schema 键面渲染器一个都不读 —— tasks / viewMode / object / startField 全零命中,照抄即空图 #5057, plugin-dashboard 的dashboardComponents用组件类名作键(全仓唯一一个),README 教的手动注册循环因此注册 11 个没人写的 type、8 个真 type 一个没注册 #5064, skills/objectui/guides/schema-expressions.md 的「Data binding withbind」示例用data-table+bind,而data-table全篇不读bind—— 照抄即空表 #5126), every one found by a human or agent probe and none by a check. Not speculative. - ③ Agent-error resistance — the strongest axis. Docs teaching a shape the code rejects is precisely what an AI author copies, and plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047 measured that the old form is rejected, not merely unsupported.
- ④ Startup scope discipline — decisive: this is consolidation of code that already exists three times over, not new capability. docs(plugin-report): README 按真实导出面重写三处虚构导出与陈旧幸存清单 #5053, docs(plugin-report): rewrite README export snippets against the real signatures #5060 and plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047's PR each hand-rolled it and each found defects the reviewer had not listed. The cost is already paid and measured; what is missing is that it runs once, in CI, instead of three times, by hand, when someone happens to think of it.
The card's own sentence is the argument: three independent reimplementations of the same harness is itself the signal that it belongs in
scripts/. I agree, and that is a PM call rather than a maintainer one.⚠️ Constraints for the implementing seat:- Verify against a planted instance, not today's tree. Every listed defect has now been fixed; a green run proves only that the tree is clean. Plant one of the four plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047 falsehoods and show the gate red.
- Resolve against the built
dist/index.d.ts, not source — that is the surface a consumer sees, and plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047's probe carried a resolution self-check proving it read artifacts. Keep that check. - Carry plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047's planted-sentinel control (an obviously-nonexistent export must produce a diagnostic), so a probe silently resolving everything to
anycannot pass as green. - Beware the false red: plugin-report docs still teach the pre-9.0 query form as the live authoring shape — and name three renderers removed at the ADR-0021 cutover #5047 measured that parse errors in one extracted file suppress semantic checking program-wide, which reads as "ablation went red" while proving nothing. The harness must distinguish syntax failure from semantic failure in its output.
NOT ruled: shape 1 (
safeParseagainst the spec schema)Left as
finding. It is strictly stronger —unrecognized_keyscatches what a type annotation cannot — but it rests on an unsolved design question the card states honestly: how a complete document is distinguished from a deliberately partial fragment. Plenty of blocks are legitimately partial, and guessing that boundary produces either false failures or a gate people learn to ignore.That is new capability with an open design question, so it is a maintainer call on scope and I am not making it. Shape 2 does not foreclose it; it makes it cheaper, because the extraction half is the same half.
Adjacent
#5106 is a second gap in this same gate (its scan surface stops at code fences and never judges the namespace half). Whoever takes this should read both before starting — two gaps in one gate are likelier to share a fix than to need two.
Generated by Claude Code
Claimed — PM dispatch.
- Session:
session_01NYgmGheCzM6NrHZN436Cxf - Branch:
claude/issue-5138-doc-snippet-compile-gate - Container & model:
claude-opus-5. No path-derived mandate in this repo, and clause ② checked rather than skipped: it does not fire. A CI gate changes what the repository accepts from its authors, not what the platform accepts from its consumers — no published type moves, no product contract's accept/reject behaviour changes. Recording the check because "no path mandate" is a floor, never a clearance, and this card sits close enough to contract work to be worth stating.
Dispatching shape 2 only, as ruled above. Shape 1 (
safeParseagainst the spec schema) is explicitly out of scope and stays a finding — its fragment-marking question is unresolved and is a maintainer call.The four constraints in the ruling are the acceptance criteria, and the first is the one that decides whether this gate is worth anything: it must be verified against a planted instance, not today's tree. Every defect this card lists has been fixed since. A green run over the current tree proves the tree is clean and nothing else.
Generated by Claude Code
- Session:
{ "issue": 5138, "status": "done", "branch": "claude/issue-5138-doc-snippet-compile-gate", "pr": "https://github.com/objectstack-ai/objectui/pull/5161", "premise_still_valid": true, "summary": "Shape 2 only, as ruled. scripts/check-doc-snippet-types.mjs extracts every fenced ts/tsx block from content/docs/**.mdx and packages/*/README.md and compiles it --strict, in isolation as its own module, against each package's built dist/*.d.ts; wired as its own unfiltered workflow (.github/workflows/doc-snippet-types.yml) whose build step is FILTERED to the packages the covered documents import, via the gate's own --build-filter, so it is not the per-PR full-repo build the 2026-08-16 #4846 ruling rejected. Coverage is declared: a document is covered unless UNGATED_DOCS names it with a reason (default covered, so a new page is gated on arrival; entries are re-derived and can only shrink) — 13 documents / 67 blocks covered today, 44 on the debt ledger with their measured diagnostic mix. Shape 1 (safeParse against spec schemas) was NOT built and is untouched; the gate's header states that exclusion plus the two others (type-literal registration, #5151 shell examples) out loud, because 'the gate looked at the one key that was correct' is the failure mode the card names.", "tests": "Union re-derived from the actual changed paths and run at the FINAL commit bf7cce684 (verified: git status clean, git diff HEAD empty): check-doc-snippet-types 0 ('67 of 67 block(s) judged, 0 failed'); check-doc-component-types 0 (481 registered, 88 exempted); check-doc-links 0; check-control-bytes 0 (4574 files); check-changeset-presence 0 ('0 of them under the src/ of a package the release covers ... no changeset is owed'); check-changeset-no-major 0; check-lint-coverage 0 (46/46); check-type-check-coverage 0 (41/41); check-skills-paths 0; pnpm type-check:scripts 0; vitest --project unit scripts/__tests__ 0 (54 files, 1242 passed); eslint on the 4 changed script files + pnpm lint:root 0 errors; pnpm --filter @object-ui/site exec fumadocs-mdx 0 (validates the new MDX markers). PLANTED-INSTANCE verification, gate committed first so each restore is a git checkout of one path, direction predicted before each leg: Leg A (all four #5047 falsehoods verbatim) -> RED, semantic only, \"content/docs/plugins/plugin-report.mdx:67:15 TS2305: Module '\\\"@objectstack/spec/ui\\\"' has no exported member 'ReportInput'.\" and the key-level errors MASKED exactly as predicted (one diagnostic, not four). Leg B (repurposed key, no masking annotation) -> \"plugin-report.mdx:73:13 TS2322: Type '{ field: string; aggregate: string; }' is not assignable to type 'string'.\" — objectName/groupingsDown produced nothing here because TS reports one excess property per literal and the columns error preempted it, isolated rather than assumed in Leg C -> \"plugin-report.mdx:73:3 TS2353: Object literal may only specify known properties, and 'objectName' does not exist in type ...\". Leg D (fabricated export) -> \"plugin-report.mdx:67:10 TS2305: Module '\\\"@object-ui/plugin-report\\\"' has no exported member 'registerDrillHandler'.\" Leg E (syntax vs semantic, #5047's exact bare-filter prose shape) -> \"[syntax] plugin-report.mdx:63:42 TS1128\" plus \"Syntax phase: 1 block(s) failed to parse and were NOT semantically checked / Semantic phase: 67 of 68 block(s) judged, 0 failed / NOTE: this run's semantic result covers 67 block(s) only. A syntax failure is not a semantic pass.\" RESTORED: all legs reverted, gate exit 0. CONTROL ABLATIONS (each control broken deliberately, then reverted): sentinel pointed at a real export -> 'HARNESS CONTROL FAILED ... the planted sentinel produced no TS2305 — the program is resolving everything to any and would report green forever'; resolution mapped to src -> 'resolution did not land on a built artifact (.../packages/types/src/index.ts)' + '39 source file(s) under a package's src/ entered the program'. Every normal run prints the three controls, including the #5047-style line \"Module name '@object-ui/types' was successfully resolved to '.../packages/types/dist/index.d.ts'\".", "open_questions": [], "out_of_scope_findings": [ "filed as #5160: five published READMEs import symbols their packages do not export — 15 TS2305 measured by the new gate over the ungated corpus (@object-ui/core x6, components x3, app-shell x2, react x1, plus objectos-integration.mdx x3); the #5016/#5048 defect class live in four more packages", "not filed, reported per dispatch: #5106 is NOT subsumed by this gate and the reason is structural — its scan surface is the same fenced-code surface, so a markdown key table is outside this gate too, and a registry namespace is a runtime string value no type checks. Its fix belongs where the card puts it (widen the first gate's scan to that one table row, or generate the tables from deriveRegistryKeys)", "not filed, a design limit stated in the gate header: an ungated document is not compiled and not counted, so the ledger is debt with names rather than a coverage claim. A per-file failure count was deliberately not added — producing one means building every package on every run, which the 2026-08-16 #4846 ruling rejected", "corrected rather than filed: content/docs/plugins/plugin-calendar-view.mdx does not resolve @object-ui/plugin-calendar-view because it is a MIGRATION GUIDE quoting the retired package on purpose. The first draft of the ledger called it a real defect and it would have been a false finding; the entry now says so, and the marker's stated meaning was broadened to cover blocks deliberately about code that no longer exists" ] }
Generated by Claude Code
- added a commit that references this issue
on Aug 18, 2026
Observation-class finding — a structural gap, not a defect anyone is stepping on today (the known instances get filed and fixed individually). Found while implementing #5047.
The gap
scripts/check-doc-component-types.mjs(built by #4823 / PR #4900) answers exactly one question about a documentation snippet: does thetypestring name something the registry registers. Its own header says so, and says what it deliberately leaves out:#4823 is closed as completed. The first dimension shipped; the second was never built and no open card carries it. So the class is currently untracked.
What that lets through — measured on #5047
Both documents in #5047 taught the pre-9.0 report query form for the entire interval since the ADR-0021 cutover. Every relevant gate was green on that prose the whole time. Baseline, run on unmodified
origin/mainat958d757before any edit:The teaching snippets contained, simultaneously:
objectNameandgroupingsDown— keys the strictReportSchemarejects outright (unrecognized_keys);columns: [{ field, aggregate }]— a repurposed key, nowstring[], so the old spelling is a type error rather than an unknown one;import type { ReportInput } from '@objectstack/spec/ui'— a type the spec does not export at all;registerDrillHandler— a fabricated export of@object-ui/plugin-report.None of it is visible to any gate, because the
typeliterals themselves (summary,matrix,joined) were fine. The gate looked at the one key that was correct.This is also why the defect class keeps recurring one document at a time: #4600 (dashboard examples), #5016 / #5048 (this package's README exports and signatures), #5047 (this schema shape), #5057, #5064, #5126 — each found by a human or agent probe, never by a check.
Shapes this could take, deliberately not chosen here
safeParsed against the matching@objectstack/specschema, andunrecognized_keysfails the gate. Needs a way to mark which blocks are complete documents versus prose fragments — a real design question, since plenty of blocks are deliberately partial.tsc --strictprobe against the builtdist/index.d.ts, and each found defects the reviewer had not listed. Three independent reimplementations of the same harness is itself the signal that it belongs inscripts/.Which of these, and how fragments get marked, is a maintainer call on scope and cost; that is why this is
findingand notpm:queue.Reference
content/docs/**.mdx代码块)没有 catalog 侧 #4616 那样的「不许出现 OBJUI-001」棘轮,同一缺陷已复发三次 #4823 / PR chore(scripts): 教学面 prose 的 OBJUI-001 棘轮 —— 文档代码块的 type 必须是注册键 #4900 — the first dimension, and the sentence deferring the second