Skip to content

Restructure the API reference to scale with releases and targets #94

Open

Description

@davidalucas

Problem

The API reference grows with the product of language families, Briosa releases, SpatialAnalyzer (SA) targets, and MP methods. The site has crossed the limits of the hosted build runner. The deploy for #92 (Server 0.9.2) cannot build on ubuntu-latest; #93 has the measurements.

This structure comes from the approved design in #75, implemented by #76. That design calls for "an individual static page for each MP method and documented release/SA combination". This issue proposes a structure that grows with current content rather than with every kept release. It revises some of #75's acceptance criteria, listed below.

What the Build Produces Today

Counted from plugins/api-reference/content.mjs on main at 62135ae:

Route Class Routes Notes
Exact release/SA pages (/api/<family>/sa-<target>/<release>/…) 30,321 28 contexts (4 families × 4 releases, with SA 2024 in 3 of them) × about 1,090 pages
Method history pages (/api/<family>/<group>/<method>) 4,378 One per family and method identity
Release-first aliases (/api/<family>/<release>/sa-<target>/…) 30,363 Noindex client-redirect pages, one per exact page
Pre-#76 entry routes and first-call aliases 1,349 The MP catalog links these heavily
API routes in total 66,411 Every route is a React route in the Docusaurus registry and an HTML file

What that costs:

  • Duplication. The 28,176 exact method pages contain only 8,361 distinct method bodies (30%). Most release/SA pairs repeat a contract published elsewhere, and only 802 of 4,378 method identities have a single body across every release and target.
  • Build memory. Peak Node working set was 17.3–18.2 GB in three local builds, against the 16 GB runner. Lowering the SSG worker count does not help. The main process holds the full route registry, plus about 162 MB of page JSON and 37 MB of alias JSON from the plugin. Compiling the reference content alone takes about 2 minutes.
  • Artifact. A complete build is 70,801 files (66,722 HTML): 1.8 GB uncompressed, or about 225 MB as a gzip tar. GitHub Pages documents a 1 GB maximum published-site size and a 10-minute deployment timeout. Deploys at this size have succeeded, so the limit is evidently measured against something other than uncompressed size, or is not yet enforced; we should not rely on either. Let Pages upload the large API artifact after CI checks #83 hit the deploy job's time limit while uploading 70,801 files. Cloudflare Workers static assets currently allows 20,000 files per version on the free plan and 100,000 on paid plans.
  • Client payload, site-wide. Docusaurus puts the route table in main.js, which every page loads, including the home page and the installation guide. It is now 29.8 MB raw (2.39 MB gzipped) and carries about 208,000 /api/… path strings. In addition, every API page loads a 1.7 MB manifest module (240 KB gzipped) listing every page ID in every context.
  • Growth. One new release for each of the four families adds about 17,500 routes (+26%): about 2,190 exact pages and 2,190 aliases per family. A third SA target would multiply again.

Goals

  • Build cost scales with current content, not with every kept release and target.
  • Every documented release/SA contract remains reachable and exact. History is preserved, not deleted.
  • Every URL that was ever canonical on briosa.dev, or that the site links to, keeps resolving to useful static HTML.
  • Snapshots remain inputs, never a new source of protocol or support claims. spatialanalyzer/briosa stays authoritative for contracts and support.
  • Stay within explicit budgets, so the next release does not reopen Make the Pages deploy build reliable as the site grows #93 (see Acceptance Criteria).

Non-goals: changing framework or host, which is reassessed only after this lands; changing the MP command catalog's information architecture; changing any support claim.

Options

A. Full pages for the current release; history as generated change records (recommended)

  • Fully render method pages only for the current release of each family, for every SA target that release documents. That is about 8,750 pages today. Release-specific guide, group, and overview pages stay per release, since they are small (about 1,800 in total).
  • Give current pages a release-less canonical URL, /api/<family>/sa-<target>/<group>/<method>, so a new release does not supersede thousands of URLs. That path already exists as the pre-Redesign API navigation with persistent SA context and method pages #76 entry route for SA 2024 group pages, and it keeps the same meaning.
  • Expand each method's existing history page (/api/<family>/<group>/<method>) into the record of change. Per-revision anchors replace release-pinned page URLs as the way to cite an exact historical contract. See "What History Looks Like".
  • Each release-qualified URL already published (/api/<family>/sa-<target>/<release>/…, 30,321 of them) becomes a frozen redirect stub. It points to the current page when the contract is identical, or to the history revision when it differs. The set is minted once and does not grow.
  • Trade-off: an older release's method is shown as its exact signature, parameter tables, and field-level diff on the history page, not as a standalone page with that release's full prose. Pages per new release: about 0 net, plus revisions on the history pages.

B. One page per contract revision

  • Publish one page per distinct contract of a method, keyed by the first release and target where it appeared. Every release/SA pair maps to its revision. Today that is 8,361 revision pages instead of 28,176 exact method pages.
  • Keeps standalone exact pages for every historical contract with no byte duplication.
  • Trade-offs: needs a new revision URL scheme that is harder to guess or hand-edit than release/SA paths. "Which release am I reading?" becomes a range ("Server 0.7.0–0.9.2, SA 2024 and 2026"). The target selector must map to revisions.

C. Both SA targets in one page per method and release

  • One page holds both targets' contracts, using tabs or a diff callout, with differences stated explicitly. This roughly halves the page count. It can combine with A or B.
  • Trade-off: conflicts with Redesign API navigation and publish version-aware MP method pages #75's criterion "static HTML and metadata identify the exact target", and the target-qualified search records need rework. It also keeps growing with every release, so on its own it is not enough.

D. Render the reference from published protocol descriptors

  • Generate the gRPC reference from a descriptor set published with each Briosa Server release (and client metadata for .NET, Python, and JavaScript). Compute diffs from structured data rather than by parsing Markdown tables, as differenceSummary does today.
  • Strengthens the source-of-truth boundary, and makes history and change records cheap and exact.
  • Trade-off: needs spatialanalyzer/briosa and the client repositories to publish machine-readable artifacts per release and target, which is a cross-repository issue. This is a later phase on top of A or B, not a replacement for them.

E. Keep the #75 URL model, but generate pages outside Docusaurus's route registry

  • Render API pages as plain static HTML in a separate build step, so memory scales per page rather than with the registry.
  • Keeps Redesign API navigation and publish version-aware MP method pages #75 unchanged. But it gives up shared layout and SPA navigation for API pages, unless they are reimplemented. It keeps about 66,000 or more files, which still runs into the Pages size, upload, and file-count limits. It also keeps the duplicate-content SEO problem. Not recommended for pages, though the same technique fits redirect stubs (see below).
A B C D E
Pages today about 15,000 about 14,500 about 17,000 as A or B about 66,000
Growth per release about 0 net changed methods only about 4,400 as A or B about 17,500
Exact historical prose Signatures and tables only Full Full Full Full
Static URL per release/SA Redirect stub Redirect stub Yes, single page as A or B Yes
Effort Medium Medium–high Medium High, cross-repo Medium

URL Stability and Redirects

URL Class Ever Canonical on briosa.dev? Proposal
Release-qualified SA-first pages (/api/<family>/sa-<target>/<release>/…) Yes, since 2026-09-23 Frozen redirect stub: to the release-less current page when the contract is identical, otherwise to the history revision
Method history /api/<family>/<group>/<method> Yes Unchanged; becomes the change record
Pre-#76 group and guide paths, with #method anchors Yes; the MP catalog links them thousands of times Keep as static stubs. Fragment-to-method mapping needs JavaScript, and the no-JavaScript links stay
Release-first method aliases (about 28,500) No. They appeared only in #76's pre-merge branch and were never canonical on main; since 2026-09-23 they have existed only as noindex redirect pages Retire, or keep only as stubs outside the registry
Release-first group and guide paths that coincide with pre-#76 versioned-docs paths Yes Keep (they are pre-#76 entry routes)

Redirect stubs should be static files written in postBuild, not React routes, so they cost disk rather than registry memory. GitHub Pages has no server-side redirects, so each kept URL is one small file: up to about 31,700 files if every published release-qualified and pre-#76 URL is kept. That set is frozen, because release-less canonical URLs stop each release from superseding more. If file count or upload time becomes the constraint, the alternative is a client-side lookup from 404.html for the release-qualified stubs. That keeps links working with JavaScript but returns HTTP 404, which is acceptable only for URLs that were never externally significant. A later move to Cloudflare could replace most stubs with a few pattern rules, but #fragment mapping still needs a client-side stub. Stubs keep noindex, a rel=canonical link to the destination, an immediate meta refresh, and a no-JavaScript link, which matches today's Legacy.tsx behavior.

Search and SEO

  • Today the sitemap lists every available exact page across all releases and targets, about 30,000 near-duplicate URLs. Google collapses duplicates on its own and spends crawl budget on them, and the Algolia index carries the same duplication.
  • Proposal: index the current release's pages for each target, plus the history pages. Older-release URLs become redirect stubs that are excluded from the sitemap. An instant meta refresh with a canonical link is treated as a permanent redirect, so existing signals move to the destination.
  • Search records stay target-qualified for current pages. History pages carry release and target facets, so "RunCribSheet SA 2024" still finds the historical contract.
  • The canonical changes trigger a recrawl. Report hosted crawler and search-console verification separately, as Redesign API navigation and publish version-aware MP method pages #75 requires.

What History Looks Like

Each method history page /api/<family>/<group>/<method> gets:

  1. An availability matrix of releases × SA targets: documented, unavailable, or not yet published. It must make no claim that SA removed a command.
  2. Revisions, newest first. Each revision covers a release range and targets, for example "Server 0.7.0–0.9.2 · SA 2024.1.0508.5 and 2026.1.0529.7". Releases that share a contract share a revision, as Servers 0.9.1 and 0.9.2 already share 0.9.0's snapshot.
  3. Per-revision content: the exact signature code block, the request and response tables, and a field-level diff from the previous revision (added, removed, renamed, type change, default change). It links to the released source at the matching Briosa tag.
  4. Stable anchors per revision, such as #server-0.7.0-sa-2024.1.0508.5, so redirect stubs and external links can land on the exact revision.

A per-release "API Changes" page for each family can be generated from the same data.

Revisions to #75's Acceptance Criteria

#75 Criterion Proposed
Individual static page for each MP method and documented release/SA combination An individual static page for each MP method in the current release for each documented SA target. Older combinations are revisions on the method's history page, reachable through a static URL stub (A) or a revision page (B)
Retain historical methods even when absent in newer targets Unchanged: the history page plus explicit unavailable states
Migrate existing group anchors and legacy paths Unchanged for every URL that was ever canonical or that the site links to. Release-first method aliases that were never canonical are explicitly retired
Target-qualified search records and canonical sitemap entries Scoped to current-release pages and history pages
Static HTML and metadata identify the exact target Unchanged for current pages. History revisions identify their target and release range

AGENTS.md ("API references publish individual method pages with static release/SA URLs…") and docs/contributing/documentation.md need matching updates.

Proposed Phasing

  1. Unblock the Server 0.9.2 deploy. This is tracked in Make the Pages deploy build reliable as the site grows #93 and is separate from this issue.
  2. Move redirect stubs out of the route registry. Write legacy and alias entry routes as static files in postBuild. Retire release-first method aliases, if approved. Measured on its own (aliases not registered), this cut main.js from 2.39 MB to 1.47 MB gzipped, but peak memory fell only to 16.13 GB. The memory reduction comes from phase 3.
  3. History data. Compute revisions from the snapshots (the plugin already hashes contracts), and render the history pages and per-release change pages.
  4. Switch older releases to history. Redirect older exact URLs, update the sitemap and search records, the tests, AGENTS.md, and the contributing guide.
  5. Descriptor-driven generation (D). Open a spatialanalyzer/briosa issue for published descriptor artifacts first.
  6. Reassess framework and host, only if the budgets below still cannot be met.

Acceptance Criteria

  • Measured in CI, the production build's peak memory is at most 10 GB on ubuntu-latest, and the build takes at most 15 minutes.
  • The site-wide main.js is at most 750 KB gzipped (the route table costs about 32 bytes gzipped per route, so about 15,000 routes is roughly 700 KB), and no API data module loaded by every API page exceeds 50 KB gzipped.
  • Excluding the frozen compatibility redirect documents, the Pages artifact stays under 20,000 files and 800 MB. As implemented in Publish a current API reference per SA target with visible history #95 it is 17,028 files and 719 MB.
  • Redirect documents total under 50 MB (44 MB in Publish a current API reference per SA target with visible history #95). Each stays under 2 KiB, except former group addresses: they carry one no-JavaScript link per method anchor, as Redesign API navigation and publish version-aware MP method pages #75 requires, and stay under 80 KiB (largest about 62 KiB in Publish a current API reference per SA target with visible history #95).
  • Adding a release to every family adds no redirect documents. It does add about 512 pages: 64 for each superseded release/SA context, covering its guides, group notes, and first-call walkthroughs. With their data modules that is roughly 1,000 files and 30 MB, so the 800 MB budget allows about two release cycles before the size work in the note below is needed. GitHub Pages' published-site limit is 1 GB.
  • Every URL class marked "keep" above resolves to static HTML with the correct canonical link, and this has regression coverage.
  • Every documented release/SA contract is reachable with its exact signature and tables.
  • npm ci and npm run check pass on Node.js 24.

Revised 2026-10-04 during review of #95. The original budgets (300 MB excluding redirect documents, 2 KB per redirect, and "a few hundred files" per release) were set before any build was measured, and #95 does not meet them. Most of the size is page chrome repeated on about 14,400 API pages: the sidebar is about 15 KB a page. The main options for reducing it later are rendering API sidebars only in the browser (about 200 MB) and sharing one page between SA targets whose contracts are identical.

Decisions Needed

  1. Option A (recommended), B, or a combination. Should C apply on top?
  2. Is "signatures, tables, and diffs on the history page" enough for older releases, or must each older release keep full standalone prose?
  3. Should current pages move to release-less canonical URLs, with exact historical citations moving to history-page anchors?
  4. May the release-first method aliases (never canonical) be retired? Should release-qualified URLs keep a static stub each, or use a 404.html lookup?
  5. Should older releases stay indexed, or only the current release plus history?

Refs #75, #93, spatialanalyzer/briosa#276.

Activity

  1. davidalucas commented on Oct 5, 2026

    @davidalucas
    ContributorAuthor

    Revised the size acceptance criteria to measured values, at the maintainer's request during review of #95.

    Criterion Original Revised #95
    Artifact excluding redirect documents 300 MB, 20,000 files 800 MB, 20,000 files 719 MB, 17,028 files
    Redirect document size 2 KB each 2 KiB each; former group addresses up to 80 KiB; 50 MB in total 29,140 under 2 KiB; 1,081 group addresses up to about 62 KiB; 44 MB
    Growth per release of every family A few hundred files, no redirects No redirects; about 512 pages, roughly 1,000 files and 30 MB No redirects (fixture test); 64 pages per superseded release/SA context

    The original budgets were set before any build was measured. Group addresses need one no-JavaScript link per method anchor, as #75 requires, so the 2 KiB budget cannot hold for them.

    At about 30 MB per release cycle, the 800 MB budget leaves roughly two cycles before the size work is needed: for example, rendering API sidebars only in the browser (about 200 MB) or sharing one page between SA targets with identical contracts.

    The memory, main.js, URL, and contract criteria are unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions