Skip to content

fix(server): index mcp.md in the chat-searchable docs corpus - #905

Merged
WaylandYang merged 2 commits into
deeplethe:devfrom
rollroyces:fix/docs-corpus-includes-mcp
Sep 24, 2026
Merged

WaylandYang merged 2 commits into
deeplethe:devfrom
rollroyces:fix/docs-corpus-includes-mcp

Conversation

@rollroyces

Copy link
Copy Markdown
Contributor

Index mcp.md in the chat-searchable docs corpus

The backend docs corpus (crates/utopia-server/src/docs_corpus.rs) only indexed ingest.md. The frontend Docs page (web/src/pages/Docs.tsx) listed both ingest.md and mcp.md, so users reading the docs saw MCP documentation that the chat's search_docs tool could never find. A user asking the chat about MCP tokens, agent access, or write permissions would get nothing back even though the answer is in the docs.

The corpus is built into AppState.docs at server startup (state.rs:59) and queried via the search_docs tool the chat agent uses. So today: web/src/docs/mcp.md exists, is bundled into the frontend, is readable to humans — but is invisible to the chat agent.

Fix

Add mcp to ARTICLES. That's the whole code change.

Drift guard

The drift shape is "files in web/src/docs/ ↔ entries in ARTICLES". The doc-comment at the top of docs_corpus.rs already calls out the invariant (slug 必须与前端 DOCS 清单一致), but there was no test catching it. Without one, mcp.md slipped in front-end-only on whatever commit added it.

New test every_corpus_md_file_is_indexed:

  • Walks web/src/docs/, picks every *.md.
  • Builds the list of slugs ARTICLES references, mapped to {slug}.md.
  • Asserts the two lists are equal.

The invariant is on-disk → indexed (not the other way round). A new file dropped in without being indexed fails the test. This is the same shape as the_backstop_can_be_raised — a value defined in two places, drift is the bug, the test locks the invariant.

If we ever add a doc that's intentionally agent-invisible (a human-only reference page, say), that needs to be encoded as a feature flag in ARTICLES (e.g. ("changelog", "...", false) with the third element meaning indexable: bool) rather than silently dropped from the backend list. Not needed today; calling it out so the next person doesn't add an invisible flag the test can't see.

Signed-off-by: rollroyces royce@rollroyces.com

The backend corpus at crates/utopia-server/src/docs_corpus.rs only
indexed ingest.md. The frontend Docs page (web/src/pages/Docs.tsx)
listed both ingest.md and mcp.md, so users reading the docs saw MCP
documentation that the chat's search_docs tool could never find.
A user asking the chat about MCP tokens, agent access, or write
permissions would get nothing back even though the answer is in the
docs.

Fix: add mcp to ARTICLES.

Drift guard: new test every_corpus_md_file_is_indexed walks
web/src/docs/ and asserts the .md files match the slugs in ARTICLES
one-to-one. The invariant is "on-disk → indexed" (not the other way
round) so a new file dropped in without being indexed fails the test.
This is the same shape as the_backstop_can_be_raised — a number defined
in two places, drift between them is the bug, the test locks the
invariant.

Signed-off-by: rollroyces <royce@rollroyces.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One-line fix with the right drift guard: the slugs and titles match the Docs page's list, and the test walks the directory rather than the constant so a new file is the failing side. CI green. Approving and merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants