feat(ci): ratchet the authored source surface in tokens, only-down (#1183)

[os-steve]

Fixes #1183

A shrink-only ceiling on HotCRM's headline positioning claim, measured by a script instead of by hand.

What landed

• scripts/check-source-token-ratchet.mjs — new gate, sibling of check-source-hygiene.mjs (same walk, same reporting style, same exit conventions). It prints the layered measurement on every run and fails when a scope exceeds its committed ceiling.
• Wired into pnpm verify as pnpm hygiene:tokens, and as a step in both CI and Code Quality, immediately after the existing hygiene step.
• test/source-token-ratchet.test.ts — 15 cases: the measurement basis, the red paths, the advisory policy, and the CI wiring itself.
• .changeset/source-token-ratchet.md — empty frontmatter (see "Changeset" below).

Why a separate file rather than a fifth check inside check-source-hygiene.mjs: every check in that script answers "is this file wrong?" with a yes/no and a remedy the author applies to that file. This one answers "how big is the app?", carries committed state (the ceilings), and its output is a report a doc cites even when it is green. Folding it in would have meant one script with two exit meanings and a header arguing two unrelated things. It follows that file's idioms; it does not share its body.

Measurement basis (ruled, not chosen)

Maintainer rulings, 2026-08-17, quoted verbatim:

「translations + seed 肯定是不需要算 token 的」

「我觉得只需要写 业务语义 ~78k 交互层 ~40k ，这样客户更好理解，也更吸引开发者」

So the measured surface is comment-stripped, blank-stripped src/**/*.ts minus src/translations/ and src/data/, reported as:

scope	dirs
business semantics	objects flows actions hooks
interaction layer	views pages dashboards apps
other authored metadata	the residual — reported, no ceiling of its own
authored total	everything above — has a ceiling, so nothing can grow unwatched in a directory that is in neither layer

Translations and seed data are not measured at all: a fifth locale must never compete with business logic for the budget.

The anchoring reading, and the 5% buffer

Two further rulings came back during review and are implemented here.

「写实测值 ~80k / ~39k」 — the measured pair is what gets published (see the measurement note below). The basis does not change; the scanner stays exactly as it is, template-literal contents included.

「给 5% 缓冲」 — the first anchor in this PR sat flush against the reading, under 1% of headroom, and was rejected: routine work should not be interrupted, only real growth should. Each ceiling is now measured × 1.05, rounded up to the next 1,000.

The reading was retaken after merging main at d038b957 (#1189, which removed the account renewal fields, their view and their seed values), 2026-08-17 03:20 UTC:

  scope                    files   lines     chars   ~tokens
  business semantics          76   8,440   321,423    80,356
  interaction layer           33   4,708   156,334    39,084
  other authored metadata     43   1,088    55,451    13,863
  ──────────────────────────────────────────────────────────
  authored total             152  14,236   533,208   133,302

  Headline: business semantics ~80k · interaction layer ~39k · authored total ~133k

  business semantics   80,356 × 1.05 =  84,374 -> ceil 1k ->  85,000   (headroom 4,644, 5.8%)
  interaction layer    39,084 × 1.05 =  41,038 -> ceil 1k ->  42,000   (headroom 2,916, 7.5%)
  authored total      133,302 × 1.05 = 139,967 -> ceil 1k -> 140,000   (headroom 6,698, 5.0%)

The rounding step is what carries two of the three a little past 5%; it is kept because a ceiling a reader can hold in their head is worth more than a few hundred tokens of precision on a number estimated as chars / 4. The arithmetic lives in one exported place, anchor(tokens), so the header, the advisory and any future re-anchoring PR cannot drift from each other.

A red run now means the surface grew more than 5% past the last agreed claim, and the failure message says so — otherwise a breach reads as "the ratchet is too tight" when it means the buffer has been spent.

The headroom advisory had to be re-pitched, not just retuned

The previous advisory fired whenever headroom exceeded a flat 2,000 tokens and told the author to lower the ceiling. Under a deliberate 5% buffer that is roughly 4.6k on business semantics, so it would have fired on every clean run and instructed people to undo the ruling.

It is now relative and one whole buffer clear of the policy: it fires only when headroom exceeds twice the ruled buffer (10%), which means the tree has genuinely shrunk rather than merely sitting inside its allowance — and when it fires it names the ceiling anchor() would commit today, so re-anchoring is a copy, not a calculation. Kept rather than dropped because shrink-only ratchets should still tighten opportunistically; that is the half of the discipline the ruling did not touch. Today's real run prints no advisory on any scope, which is the intended quiet state.

The script measures ~80.4k where the issue table says ~78k — settled, and the basis stands

Both numbers are correct for their own rule, and the difference is entirely the string-literal case the card asked me to decide deliberately.

I reproduced the PM's line-oriented rule exactly (a line is a comment when it starts with //, * or /*, with block tracking) and got 8,288 lines / ~77,629 tokens for business semantics — the issue table's 8,288 / ~78k, to the line. The scanner in this gate gets 8,440 / ~80,356. The whole delta sits in five files under src/actions/.

The cause: action bodies ship to the QuickJS sandbox as template literals, and those bodies carry their own commentary — e.g. ACTOR_NAME_RESOLUTION_SOURCE in src/actions/global.actions.ts begins `// The acting user's DISPLAY name, …`. Those lines are string contents, spliced into shipped code at authoring time. A line matcher deletes them; this scanner keeps them.

The maintainer ruled the measured pair is what gets published (「写实测值 ~80k / ~39k」), so #1187 cites ~80k / ~39k and the citation is the command, not a transcription.

The stripping rule, and how it was verified

Stated in full in the script header. In short: // to end of line and /* … */ however far it spans are removed, including trailing comments after code; string and regex literal bodies are preserved byte-for-byte; blank lines go; leading indentation stays; ~tokens = chars / 4.

TypeScript's scanner was not available to borrow. This repo is on typescript@7.0.2, the native port, whose npm package exposes no compiler API at all:

$ node -e "import('typescript').then(m=>console.log(Object.keys(m)))"
[ 'default', 'version', 'versionMajorMinor' ]

So the scanner is hand-written, and its correctness over the whole real tree was proved with esbuild (present because the ObjectStack CLI builds with it) — if the strip had moved any byte that is not a comment, minified output would differ:

for every first-party .ts file (src, test, e2e, scripts):
  esbuild.transform(src,                { loader: 'ts', minify: true }).code
    === esbuild.transform(stripComments(src), { loader: 'ts', minify: true }).code

esbuild equivalence: 286 files, 0 divergent, 0 line-count drifts, 1495852 chars removed as comments

That is a hand run recorded here rather than a committed test, because esbuild is not a declared dependency of this repo — the same reasoning test/lint-i18n-gate.test.ts gives for its own hand-run reverse verification. The hazard classes it covers are pinned as fixtures instead: a comment inside a template literal, a regex literal containing ['"] and //, trailing comments, block comments.

Reverse verification — the gate goes red on the real tree

Re-run at the new anchor, from the committed state, with only the business ceiling lowered to 80,000 (measured 80,356), then restored with git checkout:

  ✗ business semantics is ~80,356 tokens; the ratchet ceiling is ~80,000 (over by ~356). That ceiling
    already carries the ruled 5% working buffer, so this is growth past the buffer, not routine drift.
    This surface is the app's headline claim — shrink it back …, or raise the ceiling in a PR that
    quotes a maintainer ruling approving the new number. Comments are already stripped, so deleting
    comments will not help.
✗ source token ratchet failed: 1 scope(s) over ceiling      (exit 1)

# restored:
✓ source token ratchet clean                                (exit 0)

The fixture suite covers the same direction independently (a 344KB fixture under src/objects/ drives a real exit 1 against the committed 85,000 ceiling), plus two failure modes a ratchet must never answer with green: a scope that reads as empty, and a measured directory that has disappeared.

Wiring, and why it is asserted in a test

#1169 is a live example in this repo of a workflow whose triggers never fire for the files it checks. CI has no paths: filter, so the ratchet runs on every PR to main — including the PRs that touch src/, which is the whole point. Code Quality filters on paths but includes **.ts, which covers all of src/. The step is in both. test/source-token-ratchet.test.ts asserts all three facts (both workflows invoke it, CI has no path filter, pnpm verify includes it), so a future edit that quietly unhooks the gate fails a test rather than passing silently.

Gates run locally (all at 9d140ae2, the tip of this branch)

Per #1149, a green local verify is not the full CI set, so: this is what I ran, and what I did not.

• pnpm validate · pnpm typecheck · pnpm lint · pnpm lint:i18n-gate · pnpm hygiene · pnpm hygiene:tokens — all green.
• pnpm build · pnpm test — green: 108 test files, 2,627 passed, 1 skipped (the count includes refactor(account): remove the dead account-level renewal model #1189's suite, merged in for the re-anchor).
• pnpm changeset status --since=origin/main — green (the Changeset Check step).
• Control-byte self-scan over every file this PR touches, outside the gate — clean.
• Not run locally: test:coverage thresholds (coverage is scoped to src/objects/*.hook.ts, which this PR does not touch), Playwright E2E, and CodeQL. All three run on the PR.

Scope

src/ is untouched — this gate measures that tree, it does not change it. The gate is green on today's tree, so nothing needed shrinking. The comment-slimming card (#1184) will not move these numbers, because the measure is comment-stripped; that is by design.

Changeset

Empty frontmatter, following .changeset/hygiene-control-bytes-content-and-changesets.md — the precedent for a CI-only gate in this repo. Changeset Check accepts it explicitly ("the sanctioned this PR releases nothing declaration, on par with the skip-changeset label"), and it keeps the reasoning in the tree instead of in a label. No skip-changeset label applied.

Note for whoever lands next

The anchor was taken after d038b957. #1180 (do_not_call enforcement) is still in flight against src/actions/, which is inside business semantics — but it has 4,644 tokens of headroom to work with now, so it should pass untouched rather than needing a re-anchor. If it lands first and does move the number materially, merge main and re-run: the script prints the ceiling to commit.

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hotcrm	Ignored		Aug 17, 2026 3:26am