sphinx-llm-friendly

Sphinx extension that makes documentation LLM-friendly:

• A Markdown version of every page, next to its HTML version.
• llms.txt, with a link to the Markdown version of every page.
• llms-full.txt, with the Markdown version of all pages.
• HTML pages point to their Markdown version with a <link rel="alternate" type="text/markdown"> tag, and get a button to copy their Markdown version.

Setup

1. Install it:

   pip install sphinx-llm-friendly

2. Add it to extensions in conf.py:

   extensions = [
       # …
       "sphinx_llm_friendly",
   ]

3. Build your documentation with the html builder:

   sphinx-build -b html docs docs/_build/html

   The Markdown pages, llms.txt and llms-full.txt are written next to the HTML pages. In the Markdown output, links to sites from intersphinx_mapping that serve Markdown point to the Markdown version of their pages.

Configuration

llm_friendly_exclude

List of patterns, with the syntax of exclude_patterns, of documents to leave out of the Markdown output and llms.txt. Default: [].

llm_friendly_llms_full_txt_exclude

List of patterns, with the syntax of exclude_patterns, of documents to leave out of llms-full.txt only. Default: [].

llm_friendly_llms_full_txt_max_tokens

Maximum number of tokens of llms-full.txt, counted with the cl100k_base encoding of tiktoken. Exceeding it logs a warning. None lifts the limit. Default: 200_000.

llm_friendly_llms_txt_summary

Summary for llms.txt. Default: the first paragraph of the root document.

llm_friendly_llms_txt_toctree_only

If True, llms.txt only lists documents reachable through toctrees from the root document. Default: False.

To leave content out of the Markdown output, give it the llm-friendly-exclude class, e.g. with the container directive. For a sphinx-design tab, use the class-label or class-content option of tab-item.

For the Markdown output, the only directive evaluates its expression with the llm tag instead of html, e.g. use .. only:: llm for content to include only in the Markdown output, and .. only:: not llm for content to leave out of it.

Nodes from third-party extensions that are still in the doctree when HTML is written need Markdown handlers, registered with app.add_node() as llm_markdown.