Skip to content

perf(site): compile the registry once per build, not twice - #139

Merged
maehr merged 1 commit into
textrefs:stagingfrom
maehr:fix/single-registry-compile
Sep 2, 2026
Merged

maehr merged 1 commit into
textrefs:stagingfrom
maehr:fix/single-registry-compile

Conversation

@maehr

@maehr maehr commented Sep 2, 2026

Copy link
Copy Markdown
Member

Stacks on #136. Both change the same astro.config.mjs call site. Retarget to staging once #136 lands; until then this PR shows #136's commits too.

The bug

An instrumented npx astro build calls compileRegistry twice.

Astro loads astro.config.mjs in a module graph of its own, separate from the one the pages render in. src/lib/registry.ts is therefore instantiated twice in one process, and the module-level memo is not shared between the two copies.

Compiling the real registry takes about 2.6s and materialises 86,397 references, so the second pass costs that time and a second copy in memory on every build and every dev start.

Why the obvious fix does not work

The issue suggested building the predicate lazily inside the sitemap filter callback. That does not help: the two module graphs never share a module variable, whenever the first call happens. Laziness moves the second compile, it does not remove it.

The fix

Memoise on globalThis under a registered symbol, which is the one scope both graphs agree on.

const CACHE_KEY = Symbol.for('textrefs.compiledRegistry');
host[CACHE_KEY] ??= compileRegistry();

Lifetime is unchanged — the process, exactly as the module memo was — so a dev server still needs a restart after the data/ submodule changes. No new staleness.

Measured

compileRegistry calls pages built
before 2 259,812
after 1 259,812

Both runs instrumented at the top of compileRegistry, both exit 0. The instrumentation was removed and the tree re-checked after each run.

On the test

Two module graphs cannot be simulated in node:test, so nothing here asserts the cross-graph sharing directly, and I have not pretended otherwise.

What is testable is the footgun the change creates: the fixture short-circuit must stay above the cache. If it slips below, every fixture-backed test silently compiles the real registry and leaves it on the global. src/lib/registry.test.ts fails when the two lines are swapped, and its runtime jumps from 0.3ms to 871ms — which is the real compile happening.

Verification

  • npm run verify — 123 tests pass, 259,812 pages built, all internal links valid

Closes #131

🤖 Generated with Claude Code

https://claude.ai/code/session_011HLhfpsrWxq56jnZr1gD8B

Copilot AI lite review requested due to automatic review settings September 2, 2026 20:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

src/lib/registry.ts likely introduces a TypeScript type error by using a non-unique symbol (Symbol.for(...)) as a computed key in a type literal, which can break typechecking/builds.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR addresses a build-time performance issue where compileRegistry() was invoked twice due to Astro loading astro.config.mjs in a separate module graph from page rendering. It introduces a globalThis-backed cache for the compiled registry (shared across both graphs) and (stacked from #136) refactors sitemap exclusion logic into a dedicated src/lib/sitemap.ts module.

Changes:

  • Memoize the compiled registry on globalThis via a registered Symbol to avoid double compilation per build/dev start.
  • Replace the old noindex predicate module with sitemap exclusion logic, including excluding /cite/ redirects and untranslated /de/ fallback pages from the sitemap.
  • Add tests for the sitemap exclusion behavior and for the fixture short-circuit staying above the global cache.
File summaries
File Description
astro.config.mjs Switches sitemap filter to use buildSitemapExclusion() from the new sitemap module.
src/lib/registry.ts Moves the compiled-registry memoization from a module variable to a globalThis symbol cache.
src/lib/registry.test.ts Adds regression tests ensuring fixture mode does not populate the global cache and repeated loads reuse the same registry.
src/lib/sitemap.ts New sitemap exclusion rules (draft records, /cite/ redirects, untranslated locale fallbacks) and shared draftRecordIris().
src/lib/sitemap.test.ts Adds coverage for the new sitemap exclusion rules.
src/pages/cite/[...alias].astro Adds noindex meta for /cite/ aliases that redirect to draft records.
src/lib/noindex.ts Deleted (replaced by src/lib/sitemap.ts).
src/lib/noindex.test.ts Deleted (replaced by src/lib/sitemap.test.ts).
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/lib/registry.ts
Comment on lines +16 to +17
const CACHE_KEY = Symbol.for('textrefs.compiledRegistry');
type CacheHost = { [CACHE_KEY]?: CompiledRegistry };
Astro loads `astro.config.mjs` in a module graph of its own, separate
from the one the pages render in. `src/lib/registry.ts` is therefore
instantiated twice in a single build, and the module-level memo is not
shared between the two copies. An instrumented `astro build` called
`compileRegistry` twice.

Deferring the first call does not help, because the two graphs never
share a module variable whenever it runs. Memoise on `globalThis`
instead, under a registered symbol, which is the one scope both graphs
agree on. An instrumented build now calls `compileRegistry` once.

Compiling the real registry takes about 2.6s and materialises 86397
references, so the second pass cost that time and a second copy in
memory on every build and every `dev` start.

Lifetime is unchanged: the process, exactly as the module memo was. A
`dev` server still needs a restart after the `data/` submodule changes.

Closes textrefs#131

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011HLhfpsrWxq56jnZr1gD8B
@maehr
maehr force-pushed the fix/single-registry-compile branch from 69348f4 to 1e7e47f Compare September 2, 2026 20:41
@maehr
maehr merged commit 4d1c708 into textrefs:staging Sep 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants