Skip to content

No gate reads inside a code fence in skills/objectui/ — 123 TypeScript fences and 63 JSON fences are checked by nothing #7359

Description

@os-litant

Filed out-of-scope from the #7251 skills-optimization flight (its DEFER row OUI-E-01). Unassigned.

Measured

At the flight's audit ref, skills/objectui/ carried 123 TypeScript fences and 63 JSON/JSONC fences. grep -rc "os:check" skills/objectui/ returns zero.

Every objectui gate that could read them excludes the surface, by construction rather than by accident:

Gate Why it does not reach a fence in skills/
check-skills-paths.mjs Deliberately excludes fences — "A fence is a worked example", and a tutorial fence may name a file the reader is about to create. It reads inline code spans in prose only.
check-doc-links.mjs SCAN_ROOTS has no skills row.
check-doc-component-types.mjs DOCS_ROOT = 'content/docs'; its header says so verbatim — "This gate walks content/docs and nothing else: not skills/**".
check-doc-snippet-types.mjs content/docs + package READMEs.
check-doc-fence-languages.mjs content/docs + package READMEs. It judges the fence language tag, never the contents.

Net: this repo enforces path existence in skill prose, and nothing else. A fence can import a symbol that does not exist, spell a key the renderer never reads, or restate a shape the spec changed, and stay green forever.

What already exists to copy

objectstack has packages/spec/scripts/check-skill-examples.ts, with the convention that an HTML comment containing os:check, placed on the line above a fence, opts that fence into a real type-check. A port would give objectui the same opt-in marker plus a runner over skills/**.

Why this is the highest-leverage structural addition here

It is the mechanism that stops the drift class from recurring. Two rounds of it have already been cleaned by hand — #3658 / #7094's truth sweep, and #7251's own correctness items, which included an eval asserting a mobile hook API (useSwipe / useLongPress) that has zero occurrences repo-wide. Every one of those was found by a human reading, which is precisely the economics check-skills-paths.mjs was created to end for paths.

Shape of the work

  1. Port the marker convention and the runner; opt-in, so it starts at zero red.
  2. Measure how many of the 123 fences compile as-is, and land the marker on the ones that already pass — that number is the honest starting ratchet.
  3. Decide separately whether the ratchet should be shrink-only, the way check-doc-fence-languages.mjs treats its declared-file population.

Related: #7358 (the same package's other unguarded surface — the paths gate's scan root).

Activity

  1. os-litant commented on Sep 2, 2026

    @os-litant
    CollaboratorAuthor

    Design input from implementing #7360 (PR #7408), not a request to change scope here.

    The oracle this check picks is load-bearing, and the two candidates disagree on a real row today.

    #7360's audit measured each must_contain token against guides/BASENAME.md — the guide matching the eval file's own basename. That per-guide oracle is what found its rows. But the "teach the thing" half of that card is closed by adding one bullet to guides/plugin-development.md naming PluginScope.registerComponent, and three assertions require that token: evals/plugin-development.json evals 1 and 2, and evals/page-builder.json eval 1.

    After PR #7408 the counts are:

    • registerComponent in guides/plugin-development.md: 0 to 1
    • registerComponent in guides/page-builder.md: 0, unchanged

    So evals/page-builder.json eval 1 is honest under a bundle-wide oracle (the answering agent loads the whole skill, router and all guides) and still untaught under a per-guide one. Whichever this check adopts, that row is either fine or red — worth deciding deliberately rather than discovering when the gate first runs.

    Second input: whole-token comparison changes what the check can see. #7360's body already raises this; two measured examples, both in guides/testing.md, both counting 1 rather than 0 so a substring check reports them clean:

    • create — matches create: vi.fn().mockResolvedValue({ id: '1', name: 'New' }), a mocked dataSource method, where the eval means the form mode
    • view — matches the comment // vite preview

    A third, FieldWidgetProps, matched only the head of FieldWidgetPropsSchema and is corrected in PR #7408. These three are the argument for comparing whole tokens; they are filed as rows in #7405 so they are not lost if this check lands with substring semantics.

    Third, smaller input: must_not_contain deserves the opposite treatment. #7370 deliberately moved a token there as "props": { rather than the bare word, because a bare props fails any answer that names the rule while following it. A check that validates must_not_contain entries against the guides the way it validates must_contain would get that backwards.


    Generated by Claude Code

  2. self-assigned this
    on Sep 3, 2026
  3. os-litant commented on Sep 3, 2026

    @os-litant
    CollaboratorAuthor

    Claim: skills lane seat — graded and dispatched in one write (priority:p2 + pm:dispatched; domain:skills already held). Triage is vacant in objectui and this is the lane's own programme follow-up (filed from #7251's DEFER row OUI-E-01), so the seat grades it. Value gate (#13597): it is the mechanism that ends a drift class two hand-sweeps have already paid for — passes.
    Session: session_01LraLgQVGq8egUwfYZpbYt1
    Branch: claude/issue-7359-skill-fence-check
    Worktree: objectui-issue-7359
    Clause-②: no — a gate, no contract claim. Governed: the marker landing touches skills/objectui/** ⇒ the PR stays DRAFT (ui#6866 批 #7); review requests are the seat's step.

    Scope, exactly the card's three steps with step 3 recorded not decided:

    1. Port the <!-- os:check --> opt-in convention from objectstack's packages/spec/scripts/check-skill-examples.ts (marker on the line directly above a fence) as an objectui runner over skills/objectui/**/*.md — TypeScript/TSX fences type-checked, JSON/JSONC fences parsed — with a --self-test, a scripts/__tests__/ spec, a package.json script, and a workflow modelled on skills-paths.yml (no paths / paths-ignore, for the reason that file states; its test pins that).
    2. Measure how many fences compile as-is at the branch point and land the marker on exactly those — the honest starting count, stated in the PR body. Opt-in ⇒ zero red on day one.
    3. ⛔ Whether the marked population becomes shrink-only (the way check-doc-fence-languages.mjs treats its declared files) is NOT decided in this flight — record the number, add no ratchet.

    Re-measured at objectui origin/main bf244f4, 2026-09-03T01:22Z: 112 ts/tsx/typescript fences and 56 json/jsonc fences across skills/objectui/**/*.md (the card's 123 / 63 were at its audit ref); os:check occurs 0 times. ⛔ The eval-token oracle discussed in comment 5514262423 (must_contain tokens vs the guides, whole-token vs substring) is a different check and is not this card — the seat records it separately. No guide prose is edited by this flight beyond the marker lines.


    Generated by Claude Code

  4. os-litant commented on Sep 3, 2026

    @os-litant
    CollaboratorAuthor

    Claim: dev seat (os-dev subagent of the dispatching PM session — same session id, distinct seat)
    Session: session_01LraLgQVGq8egUwfYZpbYt1
    Branch: claude/issue-7359-skill-fence-check
    Worktree: /home/user/objectui-issue-7359
    Branch point: 75e7436 (objectui origin/main at worktree creation — main moved on from the bf244f4 the seat re-measured at, so the starting population is re-measured at this sha).

    Scope taken is exactly the seat's three steps in comment 5518884588: port the opt-in marker convention + a runner over skills/objectui/**/*.md, wire CI, and land the marker on exactly the fences that pass as-is — number recorded, no ratchet. The eval-token oracle in comment 5514262423 is a different check and is not built here.


    Generated by Claude Code

  5. os-litant commented on Sep 3, 2026

    @os-litant
    CollaboratorAuthor

    os-dev-report

    {
      "issue": 7359,
      "repo": "objectui",
      "status": "done",
      "branch": "claude/issue-7359-skill-fence-check",
      "branch_point": "75e7436",
      "head_sha": "70fdb0e3fbc3e876347480c4db959a7743934523",
      "pr": "https://github.com/objectstack-ai/objectui/pull/7427",
      "pr_state": "draft — stays draft (governed skills/objectui/**, ui#6866 批 #7)",
      "population": {
        "ts": { "passing": 17, "total": 112 },
        "json": { "passing": 39, "total": 56 }
      },
      "markers_landed": 56,
      "import_resolution": "Built dist/*.d.ts, not tsconfig.base.json path mappings to src. A marked ts fence compiles --strict through the paths map check-doc-snippet-types.mjs derives from every package's own exports/types, plus that gate's declared-dependency rule for third-party specifiers; the type-check harness itself (derivePackageTypePaths, deriveDeclaredDependencyPaths, compileSnippets) is IMPORTED from that gate rather than re-rolled, so the syntax/semantics split and its four self-controls are shared. src was rejected for the reason that gate's header states: the root tsconfig maps the workspace to source, so an inherited config would judge the guides against code no reader resolves. Declared precondition: exit 2 PRECONDITION NOT MET on an unbuilt or source-typed tree, on a failed harness control, and on an EMPTY marked population — never exit 0, never exit 1. Honest edge, printed on every run as an 'Unmapped specifiers:' line: a bare specifier the repository ROOT declares still resolves through pnpm's root node_modules symlinks (in the marked population: vitest, @playwright/test); the inherited UNDECLARED control bounds transitives, which is a different leak. Recorded as follow-up 3 in the PR body, not silently absorbed.",
      "ci_wiring": "Own workflow .github/workflows/skill-examples.yml, no paths / paths-ignore, modelled on skills-paths.yml's trigger shape and doc-snippet-types.yml's install/derive-filter/build body. lint.yml was NOT the route: 'pnpm check' in this repo is 'node packages/cli/dist/cli.js check' (the product's own command), so there is no check:* aggregation to join. A second step inside skills-paths.yml was also rejected — that file's own header asks it stay a checkout plus one node call, and this gate needs a build. Classified rather than left to default into silence: 'Skill Example Check' added to REQUIRED_CONTEXTS in scripts/dependabot-merge-gate.mjs, merge_group subscribed, MUST_SUBSCRIBE_MERGE_GROUP reason added, and content/docs/guide/ci-cd-pipeline.md gained an inventory row plus a section (ci-cd-pipeline-doc.test.ts requires a heading naming every workflow file).",
      "premise_still_valid": true,
      "premise_false": [],
      "reverse_verification": [
        "LEG A marked ts fence: useObjectTranslation -> useObjectTranslationNoSuchExport in guides/i18n.md; mutation proved on disk by counting BOTH texts (removed 7 -> 0, injected -> 7); gate exit 1 with '[semantic] skills/objectui/guides/i18n.md:40:10 TS2305: Module \"@object-ui/i18n\" has no exported member' and 'Semantic phase: 17 of 17 ts fence(s) judged, 1 failed'; restored by 'git checkout HEAD -- PATH', proved by blob hash fc0d4ad31577ebc079bab867f88f49ac07f02f36 plus empty git diff HEAD.",
        "LEG B marked json fence: trailing comma injected in rules/styling.md (removed 1 -> 0, injected -> 1); gate exit 1 with '[json] skills/objectui/rules/styling.md:183 Expected double-quoted property name in JSON at position 86' and 'JSON phase: 39 fence(s) parsed, 1 failed'; restored, blob fe511a6f52568fe0f017c05f1ea4c216101ad904, git diff HEAD empty.",
        "LEG C marker adjacency: one blank line inserted between the marker and its fence in rules/composition.md (marker-followed-by-blank occurrences -> 1); gate exit 1 with 'skills/objectui/rules/composition.md:203 [orphan-marker]'; restored, blob 775ca26e706e43a32cfc7cb6bc27c126b9ea2c39, git diff HEAD empty.",
        "Restored-tree re-measure: gate exit 0, worktree clean, all three blob hashes match HEAD. trap on EXIT INT TERM, absolute paths throughout. NO build step sits between mutation and measurement — the artifact the gate reads IS the mutated markdown, so no dist can hold a stale copy; the dist-preflight rule does not apply and is not claimed.",
        "OPT-IN half measured, not assumed: on the same tree --measure reports 12 parse + 83 semantic + 17 json failures across all 168 candidates while the gate itself reports 0 failed and exits 0."
      ],
      "gates": [
        { "cmd": "node scripts/check-skill-examples.mjs", "exit": 0, "verdict": "Every marked skill example holds up against the built types. (Semantic phase: 17 of 17 ts fence(s) judged, 0 failed; JSON phase: 39 fence(s) parsed, 0 failed)" },
        { "cmd": "node scripts/check-skill-examples.mjs --self-test", "exit": 0, "verdict": "check-skill-examples self-test: 19 cases pass (marker adjacency both directions, orphans, nested illustration, json/jsonc, and the compiler legs through the real harness)" },
        { "cmd": "node scripts/check-skill-examples.mjs --measure", "exit": 0, "verdict": "Starting population — ts: 17/112 pass; json: 39/56 pass." },
        { "cmd": "pnpm exec vitest run (12 wiring/pin suites, --maxWorkers=2, under the shared verify lock)", "exit": 0, "verdict": "Test Files 12 passed (12); Tests 318 passed (318) — check-skill-examples, check-skills-paths, check-doc-snippet-types, merge-queue-reporting, dependabot-merge-gate, ci-cd-pipeline-doc, workflow-cache-save-bound, lint-workflow, scripts-type-check, entry-guard-wiring, check-doc-fence-languages, check-shell-escape-residue" },
        { "cmd": "node scripts/check-skills-paths.mjs", "exit": 0, "verdict": "check-skills-paths: OK (88/89 stated path(s) resolve across 20 guide file(s); 1 baselined)" },
        { "cmd": "node scripts/check-doc-links.mjs", "exit": 0, "verdict": "ran clean" },
        { "cmd": "node scripts/check-control-bytes.mjs", "exit": 0, "verdict": "ran clean" },
        { "cmd": "node scripts/check-shell-escape-residue.mjs", "exit": 0, "verdict": "check-shell-escape-residue: OK (5/5 root(s) resolved -- skills: 16 file(s), 196 fence(s); 207 file(s) and 1304 fenced block(s) examined in total) — the marker lines do not disturb its fence walk" },
        { "cmd": "node scripts/check-doc-fence-languages.mjs", "exit": 0, "verdict": "check:doc-fences — every TypeScript block in 225 document(s) is fenced ts/tsx/typescript, except 80 declared file(s) carrying 90 block(s) (SHRINK-ONLY)" },
        { "cmd": "node scripts/check-doc-snippet-types.mjs", "exit": 0, "verdict": "Every covered documentation snippet compiles against the built types. (Semantic phase: 417 of 417 block(s) judged, 0 failed)" },
        { "cmd": "node scripts/check-changeset-presence.mjs", "exit": 0, "verdict": "No source or published contract of a released package changed in this range, so no changeset is owed. — its own verdict decides; NO changeset added" },
        { "cmd": "node scripts/check-lint-coverage.mjs", "exit": 0, "verdict": "ran clean" },
        { "cmd": "node scripts/check-governed-queue-guard.mjs --self-test", "exit": 0, "verdict": "OK check-governed-queue-guard self-test: 132 cases pass" },
        { "cmd": "node scripts/check-governed-queue-guard.mjs --test CHANGE-SET-PATHS", "exit": 3, "verdict": "GOVERNED — the correct verdict for this change set, not a red: 'Do not flip it ready, enqueue it, or arm auto-merge. Park it as a DRAFT'" },
        { "cmd": "pnpm exec eslint --format json (2 changed lintable files)", "exit": 0, "verdict": "2 files linted, 0 errors, 0 warnings. Narrowing declared: pnpm lint is turbo run lint = each workspace package's own 'eslint .', and scripts/ has no package.json so it is in no package's lint run at all (the same structural gap scripts-type-check.test.ts records for type-check). Invariance: eslint.config.js configures NO type-aware linting (no project, no projectService anywhere in it), so this diff cannot move the verdict for any untouched file." }
      ],
      "skills_surface_budget": {
        "changed_files": 13,
        "per_file_lines_before_after": "SKILL.md 136/137; auth-permissions 345/348; data-integration 443/451; i18n 185/187; mobile 166/168; page-builder 321/331; plugin-development 442/446; project-setup 340/345; schema-expressions 511/518; testing 341/345; composition 277/278; protocol 314/320; styling 316/319",
        "whole_bundle_lines_before_after": "4356 -> 4412 (+56)",
        "note": "Every added line is one byte-identical inert marker comment. 56 insertions, 0 deletions, zero prose added or rewritten, zero fence content edited."
      },
      "follow_ups_recorded_not_taken": [
        "Shrink-only ratchet on the marked population — card step 3, explicitly deferred; number re-derivable with --measure.",
        "Bare `any` inside a marked block (objectstack's third assertion) — a new assertion rather than the convention, so not in this flight.",
        "Bound root-devDependency resolution the way the UNDECLARED control bounds transitives.",
        "Widen SCAN_ROOTS to .claude/skills/** the way check-skills-paths.mjs was widened by ui#7358."
      ],
      "out_of_scope_findings": [
        "REPORT ONLY, no card filed per dispatch. Two pin suites FAIL on a BUILT tree, and CI never sees it because Test (shard N/4) runs unbuilt: scripts/__tests__/check-sdui-registration-pins.test.ts derives 'packages/app-shell/dist/console/connect/ConnectAgentWidget.js' where it asserts the 'packages/app-shell/src/console/connect/ConnectAgentWidget.tsx' path, and scripts/__tests__/check-readme-exports.test.ts gets census.excerptsNotJudged 1 where it asserts 3. Both suites' own describe blocks claim to hold 'in BOTH build states', so this is the claim being false rather than a missing case. Unrelated to this change set, which touches zero packages/** files.",
        "REPORT ONLY. 17 of the 56 json fences in skills/objectui/ are tagged `json` while carrying `//` comments or `...` elisions (12 in guides/schema-expressions.md, 4 in rules/protocol.md, 1 in rules/styling.md). They are legitimately unmarked here, but `jsonc` would be the honest tag; check-doc-fence-languages.mjs judges fence language tags and does not scan skills/, so nothing says so today.",
        "REPORT ONLY. The root-devDependency resolution edge described in import_resolution is a property of the SHARED harness, so check-doc-snippet-types.mjs carries it too: its UNDECLARED control proves resolution did not widen to transitives, and says nothing about specifiers the repository root itself declares."
      ],
      "mcp_calls": 7
    }

    Note on the card's measured numbers: the card body states 123 TS / 63 JSON at its audit ref; the seat re-measured 112 / 56 at bf244f4 in comment 5518884588. This flight branched from 75e7436 (main had moved on again) and measured exactly 112 / 56 there, so the seat's re-measure holds at the branch point too. os:check occurred 0 times before this PR. The card's premise is intact — nothing was falsified.


    Generated by Claude Code

  6. os-litant commented on Sep 3, 2026

    @os-litant
    CollaboratorAuthor

    ACCEPT — skills-lane seat (session session_01LraLgQVGq8egUwfYZpbYt1), reviewer of record, verified against GitHub and the seat's own objectui checkout, not the report.

    • PR feat(gates): check the marked fenced examples in the skill guides #7427 exists, is DRAFT, head 70fdb0e, 20 files, +1,793 −0, one commit; auto-labels configuration · ci/cd · dependencies · tests; no changeset — check-changeset-presence.mjs's own verdict ("No source or published contract of a released package changed in this range") decides, as briefed. mergeable_state: behind (branch point 75e7436; main has moved) — a draft awaiting a human merge, which brings main in.
    • Governed surface, measured by the seat at 2026-09-03T02:00Z: the only edits under skills/objectui/** are 56 byte-identical marker lines — 0 non-marker additions, 0 deletions across the 13 guide/rule files; marker count 56 at head, 0 on the base; every marker sits directly above a fence (the seat's own adjacency walk printed nothing). No prose, no fence content, no eval touched.
    • Design accepted as the card's step 1: the opt-in os:check convention ported from objectstack's runner with strict adjacency (an orphan marker fails), fence-awareness, and the type-check harness IMPORTED from check-doc-snippet-types.mjs (built dist/*.d.ts, the consumer's surface) rather than re-rolled; exit 0 / 1 / 2 with PRECONDITION NOT MET on an unbuilt tree, a failed harness control, or an EMPTY marked population (seat spot-checked scripts/check-skill-examples.mjs:145, :248, :516, :754) — a gate that checks nothing never reports success. The honest edge (root-declared bare specifiers such as vitest still resolve) is printed on every run, not hidden.
    • Step 2, the starting population: --measure at head — ts 17 / 112, json 39 / 56 pass; exactly those 56 are marked. Step 3 (shrink-only) correctly NOT decided; the number is re-derivable.
    • CI wiring: a dedicated skill-examples.yml on pull_request / push / merge_group with no paths / paths-ignore (seat-verified), pinned by its test the way skills-paths.yml is; Skill Example Check added to REQUIRED_CONTEXTS in scripts/dependabot-merge-gate.mjs, merge-queue-reporting.test.ts and the ci-cd-pipeline.md inventory updated. The lint.yml route was correctly rejected (pnpm check there is the product's own command).
    • Reverse verification, three legs with on-disk proof: a marked ts fence broken → TS2305 naming file:line; a marked json fence broken → a parse error naming file:line; a blank line between marker and fence → [orphan-marker]; each restored by blob hash + empty git diff HEAD; the opt-in half measured (--measure finds 112 failures across all 168 candidates while the gate exits 0).
    • CI at 70fdb0e: every one of the 28 check runs is success or skipped, read at 2026-09-03T02:12Z — including the new gate's own first run, Skill Example Check success (completed 02:01:01Z), Lint, Type Check, all four Test shards, Doc Snippet Type Check, Skill Guide Path Check, Shell Escape Residue Scan, Governed Surface Queue Guard (draft — correct), Changeset Declaration.
    • Follow-ups recorded on the seat ledger, not filed now: the shrink-only decision (step 3); the bare-any assertion port; bounding root-devDependency resolution; widening to .claude/skills/**; 17 json fences that are really jsonc (comments / elisions) — a tag question no gate reaches in skills/; two pre-existing pin suites failing on a BUILT tree (check-sdui-registration-pins, check-readme-exports) that CI's unbuilt shards never see — the seat dedups before filing. The eval-token oracle (comment 5514262423) remains its own future gate, as the claim said.
    • Governed skills/objectui/** ⇒ stays DRAFT; review requested from os-zhuang + hotlong; human merge is the record. At MERGED the seat strips pm:dispatched (Fixes ⇒ the card closes).

    Generated by Claude Code

  7. os-litant commented on Sep 3, 2026

    @os-litant
    CollaboratorAuthor

    Landed — skills-lane seat (session session_01LraLgQVGq8egUwfYZpbYt1), merge patrol 2026-09-03T06:55Z (git log --grep '(#7427)' on a freshly fetched origin/main): PR #7427 merged as c92dc41 at 06:27:06Z — scripts/check-skill-examples.mjs, the Skill Example Check workflow and its REQUIRED_CONTEXTS row are on main; the card closed by the merge at 06:42:22Z (os-zhuang). pm:dispatched stripped; priority:p2 · domain:skills kept, read back. The follow-ups the flight recorded (the shrink-only decision on the marked population, the eval-token oracle as its own gate, the jsonc fences, the two pre-existing pin suites that fail only on a built tree, the bare-any assertion port, the root-devDependency resolution bound, widening to .claude/skills/**) are filed by the seat as separate cards after dedup, this check-in.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions