Repository navigation
docs site: any single-segment path containing a dot renders the homepage with 200 (soft-404 class) #12233
Description
Activity
Claim: PM loop round 1 (epic PM for #12243)
Session:session_f9f0958b-ab68-46cc-801c-216aa7ee2107
Branch:claude/issue-12233-dotted-soft-404
Worktree:objectstack-issue-12233
Domain:domain:devx
File surface:apps/docs/proxy.tsandapps/docs/app/[lang]/layout.tsx(or the[lang]page, whichever the Next 16 router actually honours) plus a regression test (stop on breach; explain in the report)
Container & model: M,mode:subagent,model: opus— no path-derived mandate; routing semantics need measuring in both dev and build modes rather than reasoning
Clause-②: no
Serial constraints cleared: none. Disjoint from #12232 (which only adds files) and from #12236 (content only)Dev claim (os-dev subagent, dispatched by the PM claim above).
- Session:
session_f9f0958b-ab68-46cc-801c-216aa7ee2107(same session as the epic PM — I am theos-devsubagent it dispatched, not a second claimant) - Branch:
claude/issue-12233-dotted-soft-404(pushed empty at start, base20b0fdb568c648374963cc13fdab89f4f74594bf) - Worktree:
objectstack-issue-12233 - Body integrity: the quoted
proxy.tsmatcher arrived intact — it is byte-identical toapps/docs/proxy.tsonorigin/main(blob4c8540cfdbe284598b68d1d48592e5431b54559b), so no reconstruction was needed.
Plan: re-measure both halves of the asymmetry on a dev server first, confirm or correct the root-cause reading, then measure the candidate fixes in both
next devand a productionnext build+next start, since the card explicitly does not assume they agree.- Session:
os-dev-report
{ "issue": 12233, "status": "done", "branch": "claude/issue-12233-dotted-soft-404", "pr": "https://github.com/objectstack-ai/objectstack/pull/12258", "premise_still_valid": true, "summary": "Both PM assumptions confirmed by measurement, not reasoning. (1) The asymmetry reproduces on both halves: /foo.txt, /ads.txt, /security.txt, /anything.html, /sitemap_index.xml, /robots.txt, /sitemap.xml all returned 200 with the full homepage; /this-page-does-not-exist correctly 404'd. (2) The root-cause reading is exactly right: the dev request log prints a `proxy.ts:` timing only for paths the proxy actually handled, and dotted paths have none, while they DO show `generate-params` (generateStaticParams from app/[lang]/layout.tsx) -- and a temporary probe in the layout printed lang=\"foo.txt\", lang=\"ads.txt\", lang=\"robots.txt\", lang=\"sitemap.xml\" against lang=\"en\" for /docs, /blog and /. The issue body's quoted matcher arrived INTACT -- byte-identical to apps/docs/proxy.ts on origin/main (blob 4c8540cfd), no reconstruction needed. (3) On assumption 3: Next 16 honours BOTH candidates in dev -- measured separately, each alone, each 404ing every dotted path with every real page unchanged. I shipped the explicit guard, not dynamicParams=false: the guard states the contract at the point it is violated, is render-mode-independent, and cannot silently degrade if generateStaticParams is later edited, whereas dynamicParams=false derives its correctness from generateStaticParams and is a segment config that also governs the nested [lang]/docs/[[...slug]] and [lang]/blog/[[...slug]] segments -- a wider blast radius than this card for no externally visible difference (both yield 404). It remains available as a follow-up optimization. proxy.ts is deliberately UNCHANGED: making its matcher rewrite dotted paths would send /llms.txt, /llms-full.txt, /og/**/*.png and /docs/**.mdx to /en/<same> where nothing matches, 404ing exactly the agent-reader endpoints the epic protects. Fix: lib/i18n.ts exports isSupportedLanguage() derived from i18n.languages; app/[lang]/layout.tsx calls notFound() before rendering when lang is not a declared locale. app/[lang]/page.tsx was NOT touched (PR #12218 owns it) and did not need to be -- the layout is the right home anyway, since it also wraps [lang]/docs/** and [lang]/blog/**. FINDING FOR #12232: its precedence assumption HOLDS, measured not assumed -- app/robots.ts and app/sitemap.ts were added temporarily, the site rebuilt WITH this guard active, and /robots.txt returned 200 text/plain and /sitemap.xml 200 application/xml (build route table: `o /robots.txt`, `o /sitemap.xml`) while /ads.txt and /foo.txt stayed 404. Metadata routes beat the [lang] catch-all; this guard does not interfere. The probe files were removed and the tree left clean. Same precedence already visible pre-fix for the existing static dotted route app/llms.txt/route.ts, which never reached [lang] in either direction. Docs-only, no changeset; `skip-changeset` added additively and read back as [\"skip-changeset\"].", "tests": "All at final commit 6f5d011 (git rev-parse --short HEAD on the pushed branch), except runs explicitly against base 20b0fdb56.\n\nBEFORE/AFTER STATUS CODES -- measured both ways in BOTH modes (the prod 'before' column is a real next build of the pre-fix tree, so 'after' reads as a change, not a state):\n path dev-before | dev-after | prod-before | prod-after\n /foo.txt 200 homepage | 404 | 200 homepage | 404\n /ads.txt 200 homepage | 404 | 200 homepage | 404\n /security.txt 200 homepage | 404 | 200 homepage | 404\n /anything.html 200 homepage | 404 | 200 homepage | 404\n /sitemap_index.xml 200 homepage | 404 | 200 homepage | 404\n /robots.txt 200 homepage | 404* | 200 homepage | 404*\n /sitemap.xml 200 homepage | 404* | 200 homepage | 404*\n /this-page-does-not-exist 404 | 404 | 404 | 404\n /docs 200 'Documentation | ObjectStack' | same | 200 | 200\n /docs/getting-started/quick-start 200 'Anatomy of an ObjectStack App' | same | 200 | 200\n /docs/getting-started/quick-start.mdx 200 text/markdown | same | 200 | 200\n /en 307 -> / | same | 307 | 307\n /en/docs 307 -> /docs | same | 307 | 307\n /blog 200 'Blog | ObjectStack' | same | 200 | 200\n /blog/protocol-first-development 200 | same | 200 | 200\n / 200 homepage | same | 200 | 200\n /llms.txt 200 text/plain | same | 200 | 200\n /llms-full.txt 200 text/plain | same | 200 | 200\n /og/docs/image.png 200 image/png | same | 200 | 200\n /og/docs/getting-started/quick-start/image.png 200 image/png | same | 200 | 200\n /api/search?query=object 200 application/json | same | 200 | 200\n * /robots.txt and /sitemap.xml 404 only because those routes do not exist yet (#12232's work); the precedence probe above shows they return 200 with their real content once they do.\n\nREGRESSION PIN -- scripts/check-docs-locale-catch-all.mjs, wired as `pnpm check:docs-locale-catch-all` in Lint & Repo Gates. It encodes ONE CONDITIONAL INVARIANT rather than a text pin: if a dotted single-segment path bypasses proxy.ts's matcher, then every top-level dynamic segment under apps/docs/app/ must reject a non-locale parameter before rendering. Requirement is on the SEGMENT class, so a future app/[slug]/ is caught too.\n - self-test: `check-docs-locale-catch-all --self-test` exit 0 -- verdict line: '12 assertions over a temp fixture (real checkApp path); every limb -- deleted guard, guard behind the return, hollowed predicate, a new unguarded segment, an uncompilable matcher -- observed FAILING, and the proxy condition observed flipping the requirement off.'\n - REAL RED AGAINST REAL DATA (not construction): run against the actual pre-fix files from 20b0fdb56 it reports 2 findings ('app/[lang]/ is a top-level catch-all and nothing rejects a non-locale `lang`...' and 'lib/i18n.ts does not export `isSupportedLanguage`'); 0 on this branch.\n - live tree: exit 0 -- verdict line 'check-docs-locale-catch-all: 1 top-level dynamic segment(s), 1 guarded; dotted paths bypass proxy.ts: true'\n\nGATES (exit codes captured before any pipe; verdict lines quoted from the gate's own output):\n pnpm --filter @objectstack/docs typecheck exit 0 (script name echoed: 'fumadocs-mdx && next typegen && tsc --noEmit'; 'Types generated successfully')\n pnpm --filter @objectstack/docs exec next build exit 0 ('Compiled successfully in 25.4s'; 'Generating static pages using 2 workers (1220/1220)')\n eslint . --no-inline-config (repo-wide, 8G heap) exit 0, zero findings\n pnpm check:nul-bytes exit 0 ('OK (scanned 6766 text file(s))')\n pnpm check:docs-redirects exit 0 (92 entries)\n pnpm check:entry-guard / check:parse-guard exit 0\n check-self-test-wired / -workflow-commands exit 0 (133 scripts; the new self-test IS run by CI)\n check-step-collectors / -aggregator-roster / -ci-filter-parity / -whole-set-label-write exit 0\n pnpm check:required-contexts / check:workflow-status-functions / check:pnpm-filter-targets exit 0\n pnpm check:cross-package-test-inputs / check:agent-test-spelling exit 0\nFamily list re-derived AFTER the edits with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack` (27 families matched over the 5 changed paths), not taken from the dispatch prompt. That run reports the tree as STALE against a fast-moving origin/main; the files it named stale (check-driver-conformance.mjs, gen-sdui-manifest.sh, os-regen-merge.sh, publish-smoke.sh) are unrelated to this change surface.\n\nNOT MEASURED (reported as such, not as red): `pnpm lint` as spelled aborts on this host with 'FATAL ERROR: Ineffective mark-compacts near heap limit' at ~4055 MB, exit 134 -- a V8 heap OOM, not a lint result. The identical sweep with --max-old-space-size=8192 is green with zero findings, which is the row above. Filed as #12261.\n\nDECLARED NARROWING: scripts/pm/os-verify-lock.sh ran in DECLARED UNLOCKED MODE on this host (macOS, no usable flock -- the shared verify lock is Linux-only). Every heavy command went through the entry point and its official disclosure wording is pasted in the PR body. VERDICT lines read 'command-exit 0 . UNLOCKED (declared)'.\n\nMUTATION/RESTORE DISCIPLINE: three temporary mutations, each confirmed on disk before measuring and each restored by `git checkout HEAD -- <path>` with a git-hash-object comparison against the HEAD blob: (a) the lang probe in layout.tsx (restored to f1c6ad457, probe grep back to 0, `git diff HEAD` empty); (b) the dynamicParams=false variant; (c) the pre-fix revert for the production reverse verification (restored to 9202a8f25 / ae20a3c32, guard grep back to 1, `git diff HEAD` empty). The #12232 precedence probe files carried a `trap ... EXIT INT TERM` and their absence was confirmed by ls + `git status --porcelain` empty.\n\nENVIRONMENT NOTE: the host disk sat at 99% (11-15 GiB free) throughout. Turbopack's dev cache logged 'No space left on device (os error 28)' and disabled persistence, and during that window two consecutive /blog requests returned a transient 404 mid-HMR-recompile. A 20-request burst afterwards was 20/20 200, and the production build -- which has no HMR -- returns /blog 200 in both the before and after runs, so this was an environment artifact, not the change. Flagged rather than swallowed.\n\nCLEANUP: dev server (39233) and all four next-start servers (39234/5/6/7) stopped by recorded PID, never by process name; all five ports confirmed free and all six PIDs confirmed gone. .next and node_modules removed, `git worktree remove` succeeded WITHOUT --force on a clean tree with 0 unpushed commits.", "open_questions": [], "out_of_scope_findings": [ "filed as #12261: `pnpm lint` aborts with a V8 heap OOM (exit 134) at the default old-space limit -- the script pins --stack-size=4000 but not --max-old-space-size, and lint.yml runs it with no NODE_OPTIONS; green with an 8G heap, so headroom rather than a live outage (label: finding, unassigned)", "already filed by another agent as #12255 (independently confirmed here, no duplicate opened): apps/docs/app/page.tsx is a shadowed route -- the proxy rewrites / to /en before it can run, so `/` is served by app/[lang]/page.tsx and the redirect('/docs') in app/page.tsx is dead code" ] }```ACCEPT — PR #12258, reviewed against the diff rather than the report.
- Shape: draft, base
main, 5 files, +465/-1.skip-changeseton,Check Changesetgreen with it. - The fix is where it belongs:
isSupportedLanguage()derived fromi18n.languages, called inapp/[lang]/layout.tsxbefore render. It also covers[lang]/docs/**and[lang]/blog/**, andapp/[lang]/page.tsxwas correctly left alone. proxy.tsdeliberately unchanged, and the reasoning is better than my card's. I listedproxy.tsin the file surface as if widening the matcher were an option; it is not. Rewriting dotted paths would send/llms.txt,/llms-full.txt,/og/**/*.pngand/docs/**.mdxto/en/<same>, where nothing matches — 404ing exactly the agent-reader endpoints the epic exists to protect. Refusing the surface I handed it is the right call and I am recording it as such.dynamicParams = falsewas measured and rejected on blast radius, not on taste: it is a segment config that would also govern the nested[lang]/docsand[lang]/blogsegments, and it derives its correctness fromgenerateStaticParamsstaying correct. Both candidates were measured to 404 every dotted path; the explicit guard states the contract where it is violated.- Cross-verification for docs site:
/robots.txtand/sitemap.xmldo not exist — both answer 200 with the homepage HTML #12232: this dev independently addedapp/robots.ts/app/sitemap.tsas throwaway probes, rebuilt with its own guard active, and measured/robots.txt→200 text/plainand/sitemap.xml→200 application/xmlwhile/ads.txtand/foo.txtstayed 404 — then removed the probes and confirmed the tree clean. Two independent agents now agree that metadata routes outrank the[lang]catch-all, and the two changes compose. That is the single most useful thing in this report. - The regression pin is a conditional invariant, not a text pin: if dotted paths bypass the proxy matcher, then every top-level dynamic segment must reject a non-locale parameter — so a future
app/[slug]/is caught too. It was run red against the real pre-fix commit and reported both real findings, which is ablation on real data rather than on a fixture.
⚠️ Landing-order note, recorded now so it is not discovered by an ejection: this PR and #12262 both edit.github/workflows/lint.ymland rootpackage.jsonto register a newcheck:*gate. They will conflict textually. Whichever enters the queue second must mergemainand re-push first — the queue rebuilds against currentmainand will eject it otherwise.CI at review time: 21 green, 0 red, 8 still converging. Ready and queue happen on full green, required or not.
- Shape: draft, base
Correction to the landing-order warning above — #12258 and #12262 do not conflict. I asserted they would without testing it.
Measured, read-only:
git merge-tree --write-treebetween the two branch heads exits 0 with no conflict, because the two new gate steps land at different line ranges in.github/workflows/lint.yml(around 1111 and 1071) and theirpackage.jsonadditions are one line each in different places. Both may be queued together; neither needs to wait for the other.The general shape of the warning was right — two PRs registering a gate in the same workflow file is worth checking — but I stated the outcome instead of measuring it, which is the failure mode this repo's own rules name. Recording it here rather than deleting the claim.
One-liner
/foo.txt,/ads.txt,/sitemap_index.xml,/security.txt— every single-segment path with a dot in it returns 200 with the full homepage. Paths without a dot 404 correctly. So the site publishes an unbounded set of duplicate homepages at exactly the URLs crawlers probe by default.Measured (local dev server, same routing in production)
Root cause
apps/docs/proxy.tsmatcher excludes dotted paths from locale rewriting:so
/robots.txtskips the proxy and falls straight through to the single-segment dynamic routeapp/[lang]/page.tsx, which matches withlang = "robots.txt"and rendersHomePage()— it never checks whetherlangis a real locale. A dotless path is rewritten to/en/<path>, becomes two segments, matches nothing, and 404s properly.Expected
[lang]accepts only the locales inlib/i18n.ts. Eitherexport const dynamicParams = falsealongside the existinggenerateStaticParams, or an explicitif (!i18n.languages.includes(lang)) notFound()inapp/[lang]/layout.tsx— whichever the Next 16 app router honours for both the static and the dev path. Verify both, do not assume.Why it matters
Soft-404s at this scale burn crawl budget and split ranking signal across junk URLs; and the two paths that matter most (
/robots.txt,/sitemap.xml) are in the affected set, so this card and the robots/sitemap card must both land for either to be worth anything.Acceptance
/foo.txt,/ads.txt,/security.txt,/anything.htmlall return 404/robots.txtand/sitemap.xmlstill resolve to their real routes (regression check against the sibling card)/docs,/docs/<any real page>,/blogunchangedSource
Found in an SEO review of the docs site (
apps/docs) run on 2026-08-25, measured against the local dev server and against production. The canonical origin ishttps://objectstack.ai— maintainer ruling recorded in #10659: