fix(docs): scope the llms.txt .mdx rule so it cannot compose into a 404 (#214)

/llms.txt stated two rules -- append .mdx to a page URL, and every page is also published under a locale prefix -- over the same 79 bullets, with nothing scoping the first to English. Composing them is what a machine reader does with an llms.txt, and it yielded /zh-Hans/docs/quickstart.mdx, which 404s.

The .mdx rule now states its own scope and names the exclusion. The Other Languages section is unchanged: those page URLs resolve, and it is the only place the file says translations exist.

The comment above LANG in llms-full.txt/route.ts claimed locale text stays reachable through the .mdx rewrite. It does not: next.config.mjs rewrites /docs/:path*.mdx and no locale-prefixed form, and llms.mdx/docs/[[...slug]] calls source.getPage(slug) with no language argument. Corrected, with a note that this belief is what reached the served file.

Verified by composing the rules from the served body: rule 1 79/79 -> 200, rule 2 79/79 -> 200 per locale, cross composition 0/79. The same probe exits 1 against the pre-change body. llms-full.txt byte-identical; llms.txt differs in one line.

Closes #210

Author: hotlong

apps/docs/app/llms-full.txt/route.ts

@@ -16,8 +16,15 @@ export const revalidate = false;
  * tree it returns is already per-locale, and it currently resolves to the
  * default language when called bare — but nothing in the API contract
  * promises that, so the argument stays explicit rather than relying on an
- * unstated default. The locale text stays reachable per page through the
- * `.mdx` rewrite and the locale URLs announced at the end of `llms.txt`.
+ * unstated default. The locale text stays reachable as HTML, through the
+ * locale URLs announced at the end of `llms.txt` — and *not* through the
+ * `.mdx` rewrite, which is English-only in both halves: `next.config.mjs`
+ * rewrites `/docs/:path*.mdx` and no locale-prefixed form of it, and
+ * `llms.mdx/docs/[[...slug]]` calls `source.getPage(slug)` with no language
+ * argument, so the default language is the only thing it can serve. This
+ * comment used to claim the opposite, and that belief reached the served
+ * `/llms.txt`, whose header told every machine reader to build
+ * `/zh-Hans/docs/quickstart.mdx` — a 404.
  */
 const LANG = i18n.defaultLanguage;
 

apps/docs/app/llms.txt/route.ts

@@ -88,6 +88,20 @@ function absoluteNode(node: Node): Node {
   return node;
 }
 
+/**
+ * The header's two rules — append `.mdx`, and the locale prefixes announced
+ * under Other Languages — are read by a machine that will compose them. So the
+ * `.mdx` rule states its own scope: it used to say "appending `.mdx` to its
+ * URL", and composed with the locale rule that licensed
+ * `/zh-Hans/docs/quickstart.mdx`, which 404s. The rewrite in `next.config.mjs`
+ * matches `/docs/:path*.mdx` and nothing else, and the route behind it calls
+ * `source.getPage(slug)` with no language, so there is no locale-prefixed
+ * `.mdx` URL to advertise and no locale text behind one.
+ *
+ * Keep any rewording composable with the Other Languages section: a scope this
+ * paragraph does not state is a URL this file promises and the site does not
+ * serve, and no gate reads prose.
+ */
 export async function GET() {
   const generator = llms(source);
   const tree = source.getPageTree(LANG);
@@ -99,10 +113,11 @@ export async function GET() {
     '',
     'This is the ObjectOS product and developer documentation, grouped by the ' +
       'sections used in the site navigation. Every page below is also available ' +
-      'as Markdown by appending `.mdx` to its URL (for example ' +
-      `\`${localeUrl(LANG, 'docs/quickstart.mdx')}\`), and ` +
+      'as Markdown by appending `.mdx` to the URL exactly as listed (for ' +
+      `example \`${localeUrl(LANG, 'docs/quickstart.mdx')}\`), and ` +
       `\`${SITE_URL}/llms-full.txt\` carries the full text of every page in ` +
-      'one file.',
+      'one file. Both are English-only: the locale-prefixed URLs under Other ' +
+      'Languages below have no `.mdx` form.',
   ];
 
   // Root-level pages (index, why, quickstart, ...) come before the section