Skip to content

fix(ci): keep derived locales off the translation worklist - #222

Merged
hotlong merged 1 commit into
mainfrom
claude/issue-216-worklist-derived-locales
Aug 26, 2026
Merged

hotlong merged 1 commit into
mainfrom
claude/issue-216-worklist-derived-locales

Conversation

@hotlong

@hotlong hotlong commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Fixes #216

zh-Hant is generated from its zh-Hans sibling by apps/docs/scripts/gen-zh-hant.mjs and committed — never translated from English. check-translations.mjs derived its locale list from apps/docs/lib/i18n.ts and treated every non-default locale identically, so --worklist emitted 65 en to zh-Hant items. docs/TRANSLATION.md calls the worklist "the entire input to a pass", so those items were instructions to translate English into Traditional Chinese: a second, divergent Chinese voice on a page that already has one.

Structure: the shared leaf module, and why the duplication convention does not cover this

Adjudicated in the dispatch as the shared route; I checked it against the tree and the tree supports it, for a reason worth recording rather than just concurring with.

The re-declaration convention is documented once, over SITE_URL in check-locale-surface.mjs:142:

Deliberately re-declared rather than imported: this file is the independent check, and a check that shares a constant with the thing it checks cannot catch that constant being wrong. A mismatch shows up as every URL landing in both unexpected-url and missing-url, which is loud and unambiguous.

That rationale carries two preconditions, and DERIVED_FROM satisfies neither:

  1. It applies to a checker and the thing it checks. check-locale-surface.mjs audits the app's sitemap; SITE_URL is a value the audited system produces, so an independent copy is a genuine oracle. Neither translation script audits the other — they are two consumers of one fact about the content pipeline. Two copies cross-check nothing.
  2. It requires the mismatch to be loud. SITE_URL drift lands every URL in two rule buckets at once. DERIVED_FROM drift is silent by construction: one script skipping Traditional in its worklist while the other still compares it against English produces no finding anywhere. It surfaces as a wasted translation pass, days later, somewhere that does not point back at either file.

So the shared module holds the fact; the small readers (locales(), walk(), localeOf(), frontmatter parsing) stay duplicated, untouched. .github/scripts/lib/derived-locales.mjs carries this reasoning inline so the next person does not have to re-derive it.

Self-test hazard verified by measurement, not by reading. tools/ci-scripts/run-self-tests.mjs scans with readdirSync without recursive, plus an isFile() filter. I wrote a throwaway .github/scripts/lib/__probe.mjs containing the literal '--self-test', ran the runner, and it was not flagged while the top-level script was; the probe was then removed. lib/ is outside the scan by construction.

Missing / Stale: excluded, with the vacuous-pass hole closed

The card called these "arguably" wrong. I excluded them, after looking at both consumers:

  • The report table goes to $GITHUB_STEP_SUMMARY (translations.yml) — a human-read status surface. The zh-Hant row was structurally identical to zh-Hans (48/17 against 48/17), because a derived page carries its source's stamp. It is a duplicate row presented as a to-do list.
  • --gate=release --require=LOCALE is the one place these numbers block.

That second consumer is why exclusion alone was not enough. Removing the rows would make --require=zh-Hant pass vacuously — a release gate that cannot go red, which is worse than no gate because someone is relying on it. So it now errors and names the locale carrying the real answer:

--require names derived locale(s): zh-Hant (generated from zh-Hans) — these carry no
stale/missing rows, so requiring them would pass vacuously. Require zh-Hans instead;
the generated output is gated separately by `gen-zh-hant.mjs --check`.

The derived row is rendered as — rather than dropped or zeroed: a zero reads as "nothing to do here", true by accident and a lie the moment the generator stops being run; an absent row hides that the locale ships at all. A footnote names the gate that does ask the question.

unstamped and orphan deliberately still cover derived locales. They are defects in a file that exists — an unreachable Traditional page whose English source was retired, or one with no provenance — not work items, and both still block. Scoping them out alongside the work verdicts would have been the quiet half of this bug.

Numbers

before after
worklist entries 505 440
worklist zh-Hant 65 0
Stale total 270 222
Missing total 235 218
self-tests run by turbo run test 3 4

Per-locale after: {de:75, es:75, fr:75, ja:75, ko:75, zh-Hans:65} — verified programmatically that no other locale moved. The two column deltas are exactly the removed zh-Hant rows (48 stale + 17 missing = the 65 worklist items).

The red fixture

check-translations.mjs gains a --self-test (13 cases). The rule under test is a negative — "no derived-locale item reaches the worklist" — which is the easy kind to pass by accident: a fixture with no derived pages, a DERIVED_FROM typo disabling nothing, a walk() that never reached the tree all read as success.

So every case is asserted against the same fixture tree surveyed twice: once with DERIVED_FROM, once with an empty map — exactly what this script did before. The second run must emit what the first must not, and the self-test fails if it does not.

Ablation confirming both exclusion points are load-bearing, each mutation confirmed on disk by anchor count before reading any result, restore proven by blob hash against the HEAD blob plus an empty git diff HEAD:

mutation self-test exit
remove if (isDerived(l, derived)) continue; 1 — stale.zh-Hant.mdx emitted
widen the missing loop back to LOCALES 1 — uncovered.zh-Hant.mdx emitted

Declared deviations from the stated file surface

Two, both reported rather than done silently:

  1. docs/TRANSLATION.md — its interim warning ("--worklist does not know about this yet: it still emits en → zh-Hant items … Skip them") is made false by this change, so per the dispatch it had to go. Replaced with what is now enforced, plus a note that orphan/unstamped still block. The worklist definition at the top of the file was also incomplete and now names the exclusion.
  2. tools/ci-scripts/run-self-tests.mjs — mechanically forced, not chosen. The moment check-translations.mjs declares --self-test, the runner's own guard fails the build: declares --self-test but is not listed in SELF_TESTED. Listing it follows the check-locale-surface.mjs precedent — self-test here, gate in the workflow — and its docblock's claim that the script "declares no self-test mode" is now false and was corrected. I also recorded that the scan is top-level only, so the next shared module does not have to rediscover it.

Gates

All on the final commit 6d14f6a, working tree clean:

gate verdict line
pnpm turbo run type-check Tasks: 1 successful, 1 total (exit 0)
pnpm turbo run build Tasks: 1 successful, 1 total (exit 0)
pnpm turbo run test --force ✓ 4 self-test(s) passed (exit 0)
check-locale-surface.mjs ✓ every advertised URL has a source file … (exit 0)
check-translations.mjs ✓ translations gate passed (exit 0)
gen-zh-hant.mjs --check ✓ zh-Hant: 73 generated file(s) match … byte for byte. (exit 0)

check-translation-output.mjs --files — exit 0, ✓ translation output gate passed (138 pre-existing finding(s) reported), the same 138 main reports. Measured against a comparison worktree at d454ccb: stdout and stderr are byte-identical (diff exit 0), which is the strongest available evidence that moving DERIVED_FROM to the shared module changed nothing there.

Exit codes were captured before any pipe throughout, and each verdict above is the line the gate printed rather than a $? reading.

Filed, not fixed


Generated by Claude Code

`zh-Hant` is generated from its `zh-Hans` sibling by
`apps/docs/scripts/gen-zh-hant.mjs` and committed — never translated from
English. `check-translations.mjs` derived its locale list from
`apps/docs/lib/i18n.ts` and treated every non-default locale identically, so
`--worklist` emitted 65 `en` to `zh-Hant` items. `docs/TRANSLATION.md` calls
the worklist "the entire input to a pass", so those items instructed the pass
to translate English into Traditional Chinese: a second, divergent Chinese
voice on a page that already has one, which the generator overwrites and
`gen-zh-hant.mjs --check` rejects in the meantime.

`DERIVED_FROM` now lives in `.github/scripts/lib/derived-locales.mjs` and is
imported by both `check-translations.mjs` and `check-translation-output.mjs`.
The scripts' small readers stay duplicated — two independent parsers of one
tree cross-check each other, and a mismatch between them is loud. Neither
property holds for this map: it is a fact rather than a reader, nothing audits
it, and a drift between two copies would surface only as a wasted pass.

`stale`, `missing` and `guide-stale` are scoped to translated locales, and the
worklist is built from those three buckets, so one exclusion point covers all
four surfaces. `unstamped` and `orphan` deliberately still cover derived
locales: those are defects in a file that exists, not work items.

Worklist 505 to 440 entries with zh-Hant at 0 and no other locale moved; Stale
270 to 222, Missing 235 to 218. `--gate=release --require=zh-Hant` is now an
error rather than a vacuous pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5zjYc2BoFV2NjKBBapC7C
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check-translations.mjs --worklist hands the translation pass 65 zh-Hant items it must not act on

2 participants