Skip to content

perf: replace MDAST tag extraction with a linear PascalCase scan - #1581

Draft
erunion wants to merge 2 commits into
nextfrom
perf/fast-tag-extraction
Draft

perf: replace MDAST tag extraction with a linear PascalCase scan#1581
erunion wants to merge 2 commits into
nextfrom
perf/fast-tag-extraction

Conversation

@erunion

@erunion erunion commented Aug 6, 2026

Copy link
Copy Markdown
Member

Summary

  • Replace full MDAST parse + runSync in mdxishTags and RMDX tags with a shared linear PascalCase scanner (scanPascalCaseTags)
  • Skip fenced/inline code (and magic blocks for MDXish); RMDX excludes builtins coerced by readmeComponentsTransformer
  • Fixes severe time/memory cost on large docs

Benchmarks

Measured with Vitest on Node, cold call + 5 steady-state calls (median). Memory deltas are process.memoryUsage() before/after the cold call (heap used / RSS).

Fixture A — real Artifactory release notes (artifactory-self-managed-releases.md, ~719KB)

Primary case that motivated this change (~95 <Callout>, ~180 <Anchor>, GFM tables).

API Metric Before After Change
mdxishTags cold latency 799 ms 1.5 ms ~547× faster
mdxishTags steady median 702 ms 1.2 ms ~585× faster
mdxishTags heap used Δ +61 MB +3.0 MB ~20× less
mdxishTags RSS Δ +58 MB +2.8 MB ~21× less
tags (RMDX) throws (invalid JSX / micromark-extension-mdx-jsx) 1.1 ms, +3 MB heap Δ after succeeds; before cannot parse this MDXish doc

Fixture B — synthetic valid MDX (~932KB, 2800× Callout/Anchor/CustomWidget/TableBlock)

Used so RMDX tags can be compared apples-to-apples (strict MDX).

API Metric Before After Change
mdxishTags cold latency 2500 ms 2.3 ms ~1068× faster
mdxishTags steady median 2568 ms 1.8 ms ~1411× faster
mdxishTags heap used Δ +125 MB +1.8 MB ~69× less
mdxishTags RSS Δ +122 MB ~0 MB ~122 MB saved
tags (RMDX) cold latency 12 486 ms 2.9 ms ~4233× faster
tags (RMDX) steady median 13 931 ms 1.7 ms ~8204× faster
tags (RMDX) heap used Δ +205 MB ~0 (GC noise) orders of magnitude less
tags (RMDX) heap used after 307 MB 56 MB ~5.5× lower
tags (RMDX) RSS after 638 MB 180 MB ~3.5× lower

Test plan

  • __tests__/lib/mdxishTags.test.ts and __tests__/lib/tags.test.ts green (including <<VARIABLE>>, nested tags, Table semantics)
  • Before/after bench on Artifactory release notes + synthetic large MDX (numbers above)
  • Confirm consumers that call tags / mdxishTags still get expected component lists for custom components

mdxishTags and RMDX tags were running full parse+runSync pipelines just to
collect component names. A fence/magic-block-aware scan is enough and keeps
large docs like release notes near-instant while preserving builtin exclusions.

Co-authored-by: Cursor <cursoragent@cursor.com>
@erunion erunion added the performance Issues with performance label Aug 6, 2026
mdxishTags no longer registers micromark extensions, so remove its FEATURES
preset and update the centralisation guard test.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Issues with performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant