Skip to content

[codex] cache markdown dependency hashes#2335

Draft
fireairforce wants to merge 1 commit into
masterfrom
codex/optimize-utoopack-perf
Draft

[codex] cache markdown dependency hashes#2335
fireairforce wants to merge 1 commit into
masterfrom
codex/optimize-utoopack-perf

Conversation

@fireairforce

@fireairforce fireairforce commented Jun 6, 2026

Copy link
Copy Markdown
Member

What changed

  • Cache markdown dependency file content hashes by file mtimeNs and size while the loader process is alive.
  • Avoid repeated synchronous readFileSync + content hashing for the same embedded markdown/demo dependency when dumi recomputes markdown loader cache keys across query modes such as frontmatter, text, demo-index, demo, and default page output.

Why

Utoopack creates multiple query modules for the same markdown file. dumi already shares in-flight markdown transform work across modes, but cache-key generation still re-read every known embed/demo dependency file whenever those modes generated their cache key. This keeps correctness because a changed dependency gets a new stat signature, while unchanged files reuse their previous hash.

Local comparison

Environment: examples/normal-utoopack, DUMI_CACHE=none, cleaning dist, server, .turbopack, .dumi/tmp*, and node_modules/.cache before each build.

Small bundled example:

  • webpack build: 18.3s / 16.3s, webpack compile phase about 4.8s
  • utoopack build before this patch: 16.8s / 18.8s, RSS about 699-931 MB
  • utoopack build after this patch: 20.5s / 19.5s, RSS about 779-870 MB

The small example is too noisy for this optimization to show a stable win.

Synthetic larger docs sample: added 120 temporary markdown pages under docs/perf-bench, each with shared embed content plus external and inline demos, then removed those files after measuring.

  • webpack build with this branch: 37.6-38.4s total, compile phase 19.4-21.2s, RSS about 1.6 GB, staticRoutes=493
  • utoopack build with this branch: 20.6-22.6s total, RSS about 0.8 GB, staticRoutes=129

Important caveat: the route counts differ. webpack statically exports individual demo routes for the generated demos, while utoopack exports the normal doc routes plus the ~demos/:id route. So the total build time comparison is useful for the current dumi behavior, but it is not a pure bundler apples-to-apples benchmark.

Dev probe: server-ready/HTML fetch on the small example was around 1.5-1.6s for both webpack and utoopack. I did not treat that as conclusive browser bundle readiness data.

I also tried a broader transformer-level optimization that reused dynamic imports and enhanced-resolve setup, but the 120-page benchmark did not improve and it exposed a race in the current rehypeRaw async ESM preloading pattern, so that experiment is intentionally not included in this PR.

Validation

  • pnpm exec eslint src/loaders/markdown/index.ts
  • pnpm exec father build
  • pnpm test

@vercel

vercel Bot commented Jun 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
dumi Ready Ready Preview, Comment Jun 6, 2026 2:14pm

@pkg-pr-new

pkg-pr-new Bot commented Jun 6, 2026

Copy link
Copy Markdown
More templates

dumi

npm i https://pkg.pr.new/umijs/dumi@2335

dumi-assets-types

npm i https://pkg.pr.new/umijs/dumi/dumi-assets-types@2335

@dumijs/vue-meta

npm i https://pkg.pr.new/umijs/dumi/@dumijs/vue-meta@2335

father-plugin-dumi-theme

npm i https://pkg.pr.new/umijs/dumi/father-plugin-dumi-theme@2335

@dumijs/preset-vue

npm i https://pkg.pr.new/umijs/dumi/@dumijs/preset-vue@2335

dumi-theme-mobile

npm i https://pkg.pr.new/umijs/dumi/dumi-theme-mobile@2335

commit: 2235547

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant