Skip to content

Commit f5a9bc2

Browse files
os-zhuangclaude
andauthored
test(spec): export-surface pins compare a build-time baseline instead of running tsc (#4796) (#7090)
Seventeen pin tests across thirteen files each built their own ts.createProgram over all sixteen entry points inside a vitest case to answer one question: which source declaration does entry X export under name Y? That resolution is now a checked-in artifact, packages/spec/export-origins/<entry>.json, and the pins compare against it. Measured on this container over the 18 affected files: aggregate test time 76.3s -> 23.3s, of which the 17 pins were 55.5s -> 0.07s. The pool grew by one full compilation per retirement PR (12 files when #4796 was filed, 18 now), and at ~3.4s per case against vitest's 5s default it rode the timeout line, ejecting six unrelated PRs from the merge queue in one night. Freshness is guarded twice, independently: check:export-origins recomputes from source and byte-compares (wired into check:generated and into lint.yml's required TypeScript Type Check job), and the pins carry a compiler-free runtime cross-check so pnpm test alone is not blind to a doctored artifact. Every retired assertion has a successor that fails under the same condition; two are strictly tighter, because the old pins matched a declaration's line number as \d+ and the successors assert the declaring file exactly. Claude-Session: https://claude.ai/code/session_01WkUFF5wo3ceX4fEpXdty3k Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2d1ddf0 commit f5a9bc2

44 files changed

Lines changed: 6466 additions & 1009 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
'@objectstack/spec': patch
3+
---
4+
5+
The export-surface pins compare a build-time baseline instead of running `tsc` (#4796).
6+
7+
Seventeen pin tests across thirteen files answered the same question — "which source declaration does entry point X export under name Y?" — and each answered it by building its **own** `ts.createProgram` over all sixteen entry points and running `getTypeChecker()` inside a vitest `it()`. That resolution is now a checked-in artifact, `packages/spec/export-origins/<entry>.json`, and the pins compare against it.
8+
9+
**The cost this removes was measured, not estimated.** On this container the thirteen affected files spent **76.3s** of aggregate test time, of which the seventeen compiler cases were **55.2s** — and the pool grew by one full compilation per retirement PR (the card counted 12 files; there were 18 by the time this was written). It was also non-deterministic in the way that matters: at ~3.4s per case against vitest's 5s default, a loaded merge-queue runner pushed them over the line six times in one night, each time ejecting a PR that had never touched `packages/spec`. Two stop-the-bleed laps raised the timeout (#4856, then #4864); neither saved a millisecond of compilation, because the compilation was never the cause — it was the material.
10+
11+
**A comparison is only as good as the thing compared, so freshness is guarded twice, independently.** `check:export-origins` recomputes from source and compares bytes; it runs inside `check:generated`, hence inside lint.yml's required `TypeScript Type Check` job, so a stale or hand-edited artifact is CI-red. And the pins carry a second guard that needs no compiler at all: every origin whose kind survives to runtime is cross-checked against the entry's real namespace object, so `pnpm test` on its own is not blind to a doctored artifact either. Type-only exports are erased at runtime and are covered by the first guard, which covers everything — two gates that fail for different reasons beat one gate that has to be believed.
12+
13+
**Every pin's claim has a successor that fails under the same condition**, and two of them are strictly tighter rather than equal: the retired pins asserted a declaration's position as `<file>:<line>` with the line matched as `\d+`, i.e. never, so the successors assert the declaring file exactly. The artifact deliberately records no line number — recording one would rewrite it on every edit that shifts a line in any `.zod.ts`, turning a comparison baseline into the repo's next merge-conflict magnet. For the same reason the ten `./contracts` exports that resolve into `ai` / `@ai-sdk/provider-utils` have their pnpm-store version and peer-hash segments normalised away.
14+
15+
Sharded per entry point, following `api-surface/` and for its reason (#5837): retirement PRs rewrite whichever entries they touched, and the merge queue rebuilds server-side where no custom merge driver runs, so two PRs retiring names on different entries must touch disjoint files.
16+
17+
**Two `createProgram` cases are deliberately left as they are.** `data/driver.test.ts` and `ui/app.test.ts` compile a single file to assert that a retired key is *unwritable in the authored type* — a different fact from export origin, which no export-surface baseline can carry. `contracts/sharing-service.test.ts` parses one file with `createSourceFile` to read TSDoc; that is a syntactic parse with no program and no checker, and costs nothing. Naming them here rather than leaving the reader to wonder why `grep typescript` still finds hits.
18+
19+
No runtime or published-surface change: this is a test and tooling change plus one new generated artifact.

‎.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ packages/spec/authorable-surface.base.json merge=os-regen
4949
packages/spec/authorable-defaults/** merge=os-regen
5050
packages/spec/json-schema.manifest/** merge=os-regen
5151
packages/spec/api-surface/** merge=os-regen
52+
packages/spec/export-origins/** merge=os-regen
5253
packages/spec/api-surface-signatures.json merge=os-regen
5354
docs/protocol-upgrade-guide.md merge=os-regen
5455
docs/audits/2026-07-unknown-key-strictness-ledger.counts.md merge=os-regen

‎.github/workflows/lint.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -891,6 +891,28 @@ jobs:
891891
- name: Check the protocol upgrade guide is regenerated with the ADR-0087 registries
892892
run: pnpm --filter @objectstack/spec check:upgrade-guide
893893

894+
# The #4796 declaration-origin baseline. `export-origins/` records which
895+
# SOURCE DECLARATION every public entry point exports under every name, and
896+
# seventeen export-surface pin tests compare against it instead of each
897+
# building their own `ts.createProgram` inside a vitest case (~55s of
898+
# compilation per CI lap, and a non-deterministic timeout that ejected six
899+
# unrelated PRs from the merge queue in one night).
900+
#
901+
# A comparison is only as good as the thing compared, so THIS STEP IS WHAT
902+
# THOSE PINS MEAN. Without it a hand-edited or stale artifact would let
903+
# every one of them report success over a fiction — the dormant-gate shape
904+
# #4642 named. It belongs in this job for the same reason `check:docs`
905+
# does: no paths filter, required status check, so it cannot go dormant on
906+
# exactly the spec-only PRs that move the export surface.
907+
#
908+
# Reads `src/` via tsx and needs no build, so it runs before the workspace
909+
# build and fails in ~3s. The self-test in front of it pins the resolver's
910+
# own two edges (a re-export must read as ONE declaration; two declarations
911+
# sharing a name must NOT) — a broken resolver would otherwise write a
912+
# confidently wrong baseline and this step would agree with it.
913+
- name: Check the export-origins baseline resolves as recorded
914+
run: pnpm --filter @objectstack/spec check:export-origins
915+
894916
# The authorable KEY surface — what a metadata author may write, which for
895917
# this platform is the third-party API. `api-surface/` records exported
896918
# names and `api-surface-signatures.json` hashes factory types as TypeScript

0 commit comments

Comments
 (0)