Skip to content

Commit 00078b7

Browse files
committed
feat(platform-objects,cli): record which source revision a generated translation leaf was filled from
Extends the #8765 Option B source-hash mechanism to the generated bundles, per maintainer ruling #12069 Option A, and corrects the false "this hole cannot occur there" note that kept them out of it. `os i18n extract --source-hashes` emits <locale>.source-hashes.generated.ts — per leaf, the digest of the source revision that leaf is still a byte copy of. withSourceFallback takes it as a fourth argument and judges objects / metadataForms alongside the hand-authored sections. Records are written only where a leaf is currently a copy of the CURRENT source, so day-one stale count is 0 by construction: 9030 translated leaves measured, 1543 recorded, 7487 left legacy-trusted. No committed bundle moved a byte. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o
1 parent af56546 commit 00078b7

16 files changed

Lines changed: 1952 additions & 28 deletions
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
---
2+
"@objectstack/platform-objects": minor
3+
"@objectstack/cli": minor
4+
---
5+
6+
feat(platform-objects,cli): record which source revision a generated translation leaf was filled from (#11671)
7+
8+
Closes the half of the sticky-translation-drift class that no value comparison
9+
could reach, under maintainer ruling #12069 Option A — by extending the existing
10+
#8765 Option B source-hash mechanism to the generated bundles rather than
11+
building a second one.
12+
13+
**The hole.** `os i18n extract --fill=default` fills gaps only: any non-empty
14+
value in a translated locale wins forever. So the ordinary sequence — extract,
15+
revise the source string, extract again — rewrites `en` and strands the previous
16+
source text in every other locale. The bundle is still in sync by key, so
17+
`check:i18n` reports OK; the leaf is still present, so `check:i18n-coverage`
18+
counts it translated. Measured on #11659 at `bbe0b17`: three locales serving a
19+
602-char superseded draft of a 411-char help string under 31 green checks. Once
20+
the source has moved, that stale fill is indistinguishable **by value** from a
21+
real translation — 2648 of 3010 leaves differ from `en`, so "untranslated AND
22+
differing from the source" describes an empty set, not a noisy one.
23+
24+
**What is new.** `os i18n extract --source-hashes` writes
25+
`<locale>.source-hashes.generated.ts` beside each generated bundle: per leaf,
26+
the digest of the source revision that leaf is **still a byte copy of**.
27+
`withSourceFallback` takes that table as a fourth argument and now judges the
28+
`objects` / `metadataForms` sections as well as the hand-authored ones, so a
29+
leaf whose source has moved underneath it serves the current source string
30+
instead of a superseded draft — the same degradation an untranslated key already
31+
produces, which is the invariance the #8765 ruling turned on.
32+
33+
The generated half needs one conjunct the hand-authored half does not: the leaf
34+
must still hold the recorded bytes. Its hash table is itself generated, so a
35+
translator cannot be asked to refresh a digest by hand the way
36+
`<locale>.source-hashes.ts` asks; without that conjunct, re-translating a stale
37+
leaf would leave the old record standing and report the fresh translation as
38+
stale forever. With it, editing the value clears the flag by itself.
39+
40+
**Behaviour on the day it lands: unchanged for every leaf.** Records are
41+
written only where a leaf is currently a byte copy of the **current** source, so
42+
every record equals the current digest and nothing is stale. Measured across the
43+
nine bundle sets: 9030 translated leaves, 1543 byte-equal to `en` (records
44+
written), 7487 differing (left with no record — legacy-trusted, per the ruling's
45+
property 1, since nothing in the tree says which revision they were made from).
46+
No committed bundle changed a byte.
47+
48+
**Scope.** `--source-hashes` is off by default and `@objectstack/platform-objects`
49+
is the one bundle set that opts in, by documenting the flag in its extract
50+
config. The other eight sets keep exactly today's behaviour and can be enabled
51+
file-by-file later; a set with no companion is entirely legacy-trusted.
52+
53+
The false "this hole cannot occur there" note that kept the generated sections
54+
out of the mechanism is corrected in `source-hash.ts`, with the measurement that
55+
falsifies it.

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
"objectui:refresh": "bash scripts/bump-objectui.sh && bash scripts/build-console.sh",
3030
"objectui:clean": "rm -rf packages/console/dist .cache/objectui-*",
3131
"lint": "node --stack-size=4000 node_modules/eslint/bin/eslint.js . --no-inline-config",
32-
"i18n:extract": "tsx packages/cli/bin/run-dev.js i18n extract packages/platform-objects/scripts/i18n-extract.config.ts --locales=zh-CN,ja-JP,es-ES --fill=default --out=packages/platform-objects/src/apps/translations",
32+
"i18n:extract": "tsx packages/cli/bin/run-dev.js i18n extract packages/platform-objects/scripts/i18n-extract.config.ts --locales=zh-CN,ja-JP,es-ES --fill=default --source-hashes --out=packages/platform-objects/src/apps/translations",
3333
"check:i18n": "node scripts/check-i18n-bundles.mjs --self-test && node scripts/check-i18n-bundles.mjs",
3434
"check:i18n-coverage": "node scripts/check-i18n-coverage.mjs --self-test && node scripts/check-i18n-coverage.mjs",
3535
"check:i18n-stale-fill": "node scripts/check-i18n-stale-fill.mjs --self-test && node scripts/check-i18n-stale-fill.mjs",

‎packages/cli/src/commands/i18n/extract.ts‎

Lines changed: 45 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,13 @@ import {
1616
emitJson,
1717
isExitSignal,
1818
} from '../../utils/format.js';
19-
import { extractTranslations, renderTranslationModule, type FillStrategy } from '../../utils/i18n-extract.js';
19+
import {
20+
extractTranslations,
21+
renderTranslationModule,
22+
renderSourceHashModule,
23+
parseSourceHashModule,
24+
type FillStrategy,
25+
} from '../../utils/i18n-extract.js';
2026

2127
const FILL_STRATEGIES: FillStrategy[] = ['empty', 'default', 'todo'];
2228

@@ -89,6 +95,12 @@ export default class I18nExtract extends Command {
8995
default: true,
9096
allowNo: true,
9197
}),
98+
'source-hashes': Flags.boolean({
99+
description:
100+
'Also write <locale>.source-hashes.generated.ts — the provenance companion that lets a stale fill be told from a translation (#11671). Off by default: it is a format addition, so a bundle set opts in by documenting the flag in its extract config.',
101+
default: false,
102+
allowNo: true,
103+
}),
92104
'dry-run': Flags.boolean({
93105
description: 'Print to stdout instead of writing to --out',
94106
default: false,
@@ -131,9 +143,25 @@ export default class I18nExtract extends Command {
131143
? declared.defaultLocale
132144
: 'en');
133145

146+
// Resolved before the extract because the previously committed provenance
147+
// records are an INPUT to it: they are the mechanism's only memory, and a
148+
// run that could not read them would silently re-derive every record from
149+
// the current tree and forget the drift it is supposed to be holding on to.
150+
const outDir = flags.out ? path.resolve(process.cwd(), flags.out) : undefined;
151+
const previousSourceHashes: Record<string, Record<string, string>> = {};
152+
if (flags['source-hashes'] && outDir) {
153+
for (const locale of locales ?? []) {
154+
const file = path.join(outDir, `${locale}.source-hashes.generated.ts`);
155+
if (!fs.existsSync(file)) continue;
156+
const table = parseSourceHashModule(fs.readFileSync(file, 'utf8'));
157+
if (table) previousSourceHashes[locale] = table;
158+
}
159+
}
160+
134161
const result = extractTranslations(normalized, {
135162
defaultLocale,
136163
locales,
164+
previousSourceHashes,
137165
fill: flags.fill as FillStrategy,
138166
filter,
139167
// Merge (the default) never overwrites an existing non-default-locale
@@ -222,7 +250,8 @@ export default class I18nExtract extends Command {
222250
return;
223251
}
224252

225-
const outDir = path.resolve(process.cwd(), flags.out);
253+
// `flags.out` is non-empty here — the two branches above return otherwise.
254+
const resolvedOutDir = outDir as string;
226255

227256
// Every file a normal run would emit, paired with its rendered content.
228257
// Both branches below iterate this, so `--check` can never diverge from
@@ -231,18 +260,29 @@ export default class I18nExtract extends Command {
231260
for (const locale of localesEmitted) {
232261
if (result.counts[locale] > 0) {
233262
emitted.push({
234-
file: path.join(outDir, `${locale}.objects.generated.ts`),
263+
file: path.join(resolvedOutDir, `${locale}.objects.generated.ts`),
235264
content: renderTranslationModule(result.bundles[locale], { locale, objectsOnly }),
236265
keys: result.counts[locale],
237266
});
238267
}
239268
if (emitsMetadataForms(locale)) {
240269
emitted.push({
241-
file: path.join(outDir, `${locale}.metadata-forms.generated.ts`),
270+
file: path.join(resolvedOutDir, `${locale}.metadata-forms.generated.ts`),
242271
content: renderTranslationModule(result.bundles[locale], { locale, kind: 'metadataForms' }),
243272
keys: metadataFormsCounts[locale],
244273
});
245274
}
275+
// The provenance companion rides in the SAME list, so `--check` compares
276+
// it by the same byte-for-byte rule as the bundles it belongs to and can
277+
// never diverge from what a real extract writes.
278+
const table = result.sourceHashes[locale];
279+
if (flags['source-hashes'] && table) {
280+
emitted.push({
281+
file: path.join(resolvedOutDir, `${locale}.source-hashes.generated.ts`),
282+
content: renderSourceHashModule(table, { locale }),
283+
keys: Object.keys(table).length,
284+
});
285+
}
246286
}
247287

248288
if (flags.check) {
@@ -269,7 +309,7 @@ export default class I18nExtract extends Command {
269309
process.exit(1);
270310
}
271311

272-
fs.mkdirSync(outDir, { recursive: true });
312+
fs.mkdirSync(resolvedOutDir, { recursive: true });
273313
let written = 0;
274314
for (const { file, content, keys } of emitted) {
275315
fs.writeFileSync(file, content, 'utf8');

‎packages/cli/src/utils/i18n-extract.ts‎

Lines changed: 111 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ import { DEFAULT_METADATA_TYPE_REGISTRY } from '@objectstack/spec/kernel';
7979
import { deriveFieldGroupLayout } from '@objectstack/spec/data';
8080
import { expandViewContainer } from '@objectstack/spec/ui';
8181
import { walkPageComponents } from '@objectstack/lint';
82+
import { collectFilledFromHashes } from '@objectstack/platform-objects/apps';
8283

8384
// ─── Public types ──────────────────────────────────────────────────────
8485

@@ -158,6 +159,18 @@ export interface ExtractOptions {
158159
* This makes extract idempotent — re-running only fills the gaps.
159160
*/
160161
mergeExisting?: boolean;
162+
/**
163+
* The `<locale>.source-hashes.generated.ts` tables already committed beside
164+
* the bundles, keyed by locale.
165+
*
166+
* This is the mechanism's ONLY memory (#11671 / #12069 Option A): a leaf that
167+
* is a byte copy of a source revision keeps its record across runs, which is
168+
* what makes the drift detectable after the source moves. Passing nothing
169+
* makes the run behave like a first extract — every record is re-derived from
170+
* the tree, so leaves that already drifted stay legacy-trusted rather than
171+
* being reported.
172+
*/
173+
previousSourceHashes?: Record<string, Record<string, string>>;
161174
}
162175

163176
export interface ExtractResult {
@@ -167,6 +180,18 @@ export interface ExtractResult {
167180
counts: Record<string, number>;
168181
/** Total expected entries before per-locale merge filtering. */
169182
totalExpected: number;
183+
/**
184+
* Per translated locale, the digest of the source revision each GENERATED
185+
* leaf is still a byte copy of — the content of
186+
* `<locale>.source-hashes.generated.ts`.
187+
*
188+
* Computed by `collectFilledFromHashes` in
189+
* `@objectstack/platform-objects/apps`, the module maintainer ruling #8765
190+
* Option B put the mechanism in; the extractor supplies the tree and the
191+
* previous records and owns none of the rule. The default locale gets no
192+
* entry: it is the source, not a copy of one.
193+
*/
194+
sourceHashes: Record<string, Record<string, string>>;
170195
}
171196

172197
// ─── Walk helpers ──────────────────────────────────────────────────────
@@ -1233,7 +1258,18 @@ export function extractTranslations(config: any, opts: ExtractOptions = {}): Ext
12331258
counts[locale] = count;
12341259
}
12351260

1236-
return { bundles, counts, totalExpected: entries.length };
1261+
const sourceHashes: Record<string, Record<string, string>> = {};
1262+
const sourceBundle = bundles[defaultLocale];
1263+
for (const locale of locales) {
1264+
if (locale === defaultLocale) continue;
1265+
sourceHashes[locale] = collectFilledFromHashes(
1266+
bundles[locale],
1267+
sourceBundle,
1268+
opts.previousSourceHashes?.[locale],
1269+
);
1270+
}
1271+
1272+
return { bundles, counts, totalExpected: entries.length, sourceHashes };
12371273
}
12381274

12391275
// ─── Serialization ─────────────────────────────────────────────────────
@@ -1310,6 +1346,80 @@ export function renderTranslationModule(
13101346
return lines.join('\n');
13111347
}
13121348

1349+
/**
1350+
* Render one locale's generated source-hash table as a TypeScript module body —
1351+
* the `<locale>.source-hashes.generated.ts` companion.
1352+
*
1353+
* Deliberately types the export STRUCTURALLY (`Readonly<Record<string,
1354+
* string>>`) instead of importing `SourceHashes`. The companion is written into
1355+
* whichever package owns the bundles, and only one of those packages can spell
1356+
* the type with a relative import; an import path guessed per package is a
1357+
* portability bug waiting for the second package to use this. The structural
1358+
* type is what `SourceHashes` is defined as, so nothing is lost.
1359+
*
1360+
* Keys are emitted sorted, and every key is quoted (they are dotted paths, so
1361+
* `formatKey` would quote them anyway). Both are load-bearing for `--check`:
1362+
* the comparison is byte-for-byte, so a table that reordered with the walk
1363+
* would fail on a tree that is in fact in sync.
1364+
*/
1365+
export function renderSourceHashModule(
1366+
hashes: Record<string, string>,
1367+
options: { locale: string; exportName?: string },
1368+
): string {
1369+
const exportName = options.exportName ?? `${camelize(options.locale)}GeneratedSourceHashes`;
1370+
const keys = Object.keys(hashes).sort();
1371+
const lines: string[] = [];
1372+
lines.push('// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.');
1373+
lines.push('');
1374+
lines.push('/**');
1375+
lines.push(` * Auto-generated by 'os i18n extract' for locale '${options.locale}'. Do not hand-edit.`);
1376+
lines.push(' *');
1377+
lines.push(" * Each entry is the digest of the SOURCE REVISION that this locale's leaf at");
1378+
lines.push(' * that path is still a byte copy of — provenance for the generated half of the');
1379+
lines.push(' * bundles (#11671, maintainer ruling #12069 Option A, extending #8765 Option B).');
1380+
lines.push(' *');
1381+
lines.push(' * An entry exists only while the leaf IS such a copy. Re-translate the leaf in');
1382+
lines.push(' * `<locale>.objects.generated.ts` and the next extract drops its entry by');
1383+
lines.push(' * itself — the table makes no claim about text a translator wrote. A path with');
1384+
lines.push(' * no entry is LEGACY-TRUSTED and never reported stale.');
1385+
lines.push(' *');
1386+
lines.push(' * ⚠️ Do not "fix" a staleness report by editing this file. Refreshing a digest');
1387+
lines.push(' * records that the current text was copied from the current source, which is');
1388+
lines.push(' * the false claim the mechanism exists to detect. Fix the TRANSLATION.');
1389+
lines.push(' */');
1390+
lines.push('');
1391+
lines.push(`export const ${exportName}: Readonly<Record<string, string>> = {`);
1392+
for (const key of keys) lines.push(` ${JSON.stringify(key)}: ${JSON.stringify(hashes[key])},`);
1393+
lines.push('};');
1394+
lines.push('');
1395+
return lines.join('\n');
1396+
}
1397+
1398+
/**
1399+
* Read a committed `<locale>.source-hashes.generated.ts` back into a table.
1400+
*
1401+
* The module body is written by {@link renderSourceHashModule}, which quotes
1402+
* every key and every value, so the object literal is already valid JSON — the
1403+
* parse needs no TypeScript and no evaluation. A file that does not parse is a
1404+
* hard `undefined` (treated as "no previous records", i.e. everything
1405+
* legacy-trusted) rather than a guess: inventing records from a file we cannot
1406+
* read is how a mechanism starts asserting provenance it does not have.
1407+
*/
1408+
export function parseSourceHashModule(source: string): Record<string, string> | undefined {
1409+
const marker = source.indexOf('export const');
1410+
const open = marker < 0 ? -1 : source.indexOf('= {', marker);
1411+
if (open < 0) return undefined;
1412+
const literal = source.slice(open + 2).replace(/;\s*$/, '');
1413+
try {
1414+
const parsed = JSON.parse(literal.replace(/,(\s*})/g, '$1'));
1415+
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return undefined;
1416+
for (const value of Object.values(parsed)) if (typeof value !== 'string') return undefined;
1417+
return parsed as Record<string, string>;
1418+
} catch {
1419+
return undefined;
1420+
}
1421+
}
1422+
13131423
function camelize(locale: string): string {
13141424
// 'zh-CN' → 'zhCN', 'ja-JP' → 'jaJP', 'es-ES' → 'esES'
13151425
return locale.replace(/-(.)/g, (_m, c) => c.toUpperCase());

0 commit comments

Comments
 (0)