fix(docs): add redirects and llms.txt improvements to prevent AI agent 404s

[michaelmagan]

Summary

• Adds 27 permanent redirects in next.config.mjs to catch common AI agent URL guesses: top-level shortcuts (/quickstart, /mcp), wrong slugs (/concepts/component-state), and folder URLs without index pages (/guides/setup-project)
• Improves llms.txt with a URL guide line telling agents to use links verbatim, and appends (section) to section headers so they aren't mistaken for URL paths
• Documents the folder index page requirement in docs/AGENTS.md

Test plan

• npm run build passes in docs/
• Verify redirects work: curl -I localhost:8263/quickstart returns 308
• Verify localhost:8263/llms.txt shows (section) suffixes and URL guide line
• Spot-check folder redirects: /getting-started, /concepts, /guides/setup-project

🤖 Generated with Claude Code

[pullfrog]

Reviewed PR #2697. All 27 redirect destinations verified against actual MDX pages — no broken targets, no source conflicts, no duplicates. Left two non-blocking comments: (1) Group C folder-index redirects should consider permanent: false since Next.js redirects run before page rendering, and (2) /reference/react-sdk-legacy/migration is a minor gap in coverage.

Task list (7/7 completed)

• Read diff and identify areas of change
• Verify all redirect destinations exist
• Check for conflicts and duplicates
• Analyze redirect correctness and edge cases
• Review llms.txt changes
• Review AGENTS.md changes
• Submit review

  ^{｜ View workflow run ｜ Triggered by Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
tambo-docs	Ready	Preview, Comment	Mar 30, 2026 5:52pm

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
cloud	Skipped		Mar 30, 2026 5:52pm
showcase	Skipped		Mar 30, 2026 5:52pm

[pullfrog]

TL;DR — Adds 27 permanent redirects and improves llms.txt output so AI agents stop hitting 404s when they guess or shorten doc URLs.

Key changes

• Add 27 redirect rules to next.config.mjs — covers top-level shortcut aliases (/quickstart, /mcp), wrong-slug synonyms (/concepts/component-state), and folder URLs that lack an index.mdx
• Harden llms.txt against URL guessing — appends (section) to section headings so agents don't treat them as paths, and adds a URL guide telling agents to use links verbatim
• Document the folder index page requirement in docs/AGENTS.md — every folder with child pages must either have an index.mdx or a redirect so the folder URL never 404s

_{Summary ｜ 3 files ｜ 1 commit ｜ base: main ← magan/fix-docs-404s-for-ai-agents}

Redirect rules for common AI agent URL guesses

Before: AI agents frequently guessed short or incorrect URLs like /quickstart, /mcp, /concepts/component-state, or bare folder paths like /guides — all returning 404s.
After: 27 permanent redirects in three groups (top-level aliases, wrong-slug synonyms, folder-index fallbacks) route these requests to the correct pages.

The redirects are organized into three commented groups in the redirects() function:

Group	Purpose	Examples
A — Top-level shortcuts	Catch common bare-path guesses	/quickstart → /getting-started/quickstart, /mcp → /concepts/model-context-protocol
B — Wrong-slug synonyms	Catch plausible but incorrect slugs	/concepts/component-state → /concepts/generative-interfaces/component-state
C — Folder-index fallbacks	Folders without index.mdx	/guides → /guides/coding-agent-skills, /reference → /reference/react-sdk

docs/next.config.mjs

llms.txt disambiguation for section headings

Before: Section headings in llms.txt rendered as ## Getting Started, which agents sometimes treated as URL paths.
After: Headings render as ## Getting Started (section), and a URL guide line explicitly tells agents to use links verbatim.

docs/src/app/llms.txt/route.ts

Folder index page policy in docs/AGENTS.md

Before: No documented requirement for folder index.mdx files — missing ones caused silent 404s.
After: AGENTS.md now requires every folder with children to have either an index.mdx or a redirect entry.

docs/AGENTS.md

  ^{｜ View workflow run ｜ Triggered by Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

All redirect destinations verified — every target path maps to an existing MDX page, no source conflicts with existing content, and no duplicate sources. Two items worth considering below.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

No action needed. The author's comment "Good catch: now 307." was acknowledging the review feedback about using permanent: false for Group C redirects — the change is already implemented in the code and the thread is resolved.

  ^{｜ View workflow run ｜ Triggered by Pullfrog ｜ Using Claude Opus ｜ 𝕏}