fix: render math as native MathML at scrape time

[ridge-kimani]

Fixes the missing math rendering closes #1264 #750

Problem

Sphinx-based docs such as scikit-learn, NumPy, SciPy and statsmodels ship their equations as raw TeX inside span.math / div.math nodes and rely on MathJax to render them in the browser. MathJax never runs in DevDocs, so pages show \(...\) and \[...\] verbatim.

Approach

Render the math to native MathML at scrape time, which is what the PyTorch pages already ship (their KaTeX output includes MathML, and the PyTorch filter kept it). MathML renders in every browser DevDocs supports without any runtime JavaScript or fonts, works offline, and follows the dark theme since it inherits the text color.

• Docs::MathRenderer (lib/docs/core/math_renderer.rb) wraps KaTeX, run through ExecJS. It renders all the expressions of a page in a single JavaScript call, keeps the TeX source as an <annotation>, and falls back to showing the escaped source when KaTeX can't parse an expression. Bare underscores in \text{...}, which MathJax tolerates, are escaped so KaTeX accepts them.
• Docs::MathFilter (lib/docs/filters/core/math.rb) is a core filter, registered before clean_html so MathJax <script> sources survive. It only touches explicit markup: Sphinx/MathJax delimiters, MathJax 2 script[type="math/tex"], MathJax 3 mjx-container (keeps the assistive MathML) and pre-rendered KaTeX (re-rendered from its TeX source, so the KaTeX HTML/CSS are no longer needed). Existing <math> is left alone. $...$ delimiters are opt-in per scraper via options[:math_dollars], because dollar signs are too common in shell and PHP docs to be safe by default.
• clean_text no longer removes empty MathML nodes (mspace, mtext, mtd, mrow), and display math gets a margin in the stylesheet.
• The now-redundant .katex-html removal was dropped from the PyTorch filter.

The katex gem goes in the docs Bundler group, so the app itself doesn't load it. Scraping already requires a JavaScript runtime supported by ExecJS (see the README), so nothing new is needed to run it.

Testing

• bundle exec rake: 829 runs, 0 failures, including new tests for the renderer and the filter (every supported markup form, the $ opt-in, the currency false-positive case, and the malformed-TeX fallback).
• Scraped the scikit-learn user guide chapters locally: 46 pages, 2,082 equations converted, 0 fallbacks, and checked the pages in the app in both the light and dark themes.
• Ran the test suite and a scikit-learn scrape inside the Docker image.

Follow-ups

This PR adds the core filter and regenerates nothing by itself, to keep the review small. Regenerating scikit-learn and the other Sphinx docs (NumPy, SciPy, statsmodels, ...) can follow in separate PRs.

Screenshots

Example in scikit_learn

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	gem/​katex@​0.11.0

View full report