[codex] cache markdown dependency hashes#2335
Draft
fireairforce wants to merge 1 commit into
Draft
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dumi
dumi-assets-types
@dumijs/vue-meta
father-plugin-dumi-theme
@dumijs/preset-vue
dumi-theme-mobile
commit: |
19c5bbe to
2235547
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
mtimeNsand size while the loader process is alive.readFileSync+ content hashing for the same embedded markdown/demo dependency when dumi recomputes markdown loader cache keys across query modes such asfrontmatter,text,demo-index,demo, and default page output.Why
Utoopack creates multiple query modules for the same markdown file. dumi already shares in-flight markdown transform work across modes, but cache-key generation still re-read every known embed/demo dependency file whenever those modes generated their cache key. This keeps correctness because a changed dependency gets a new stat signature, while unchanged files reuse their previous hash.
Local comparison
Environment:
examples/normal-utoopack,DUMI_CACHE=none, cleaningdist,server,.turbopack,.dumi/tmp*, andnode_modules/.cachebefore each build.Small bundled example:
The small example is too noisy for this optimization to show a stable win.
Synthetic larger docs sample: added 120 temporary markdown pages under
docs/perf-bench, each with shared embed content plus external and inline demos, then removed those files after measuring.staticRoutes=493staticRoutes=129Important caveat: the route counts differ. webpack statically exports individual demo routes for the generated demos, while utoopack exports the normal doc routes plus the
~demos/:idroute. So the total build time comparison is useful for the current dumi behavior, but it is not a pure bundler apples-to-apples benchmark.Dev probe: server-ready/HTML fetch on the small example was around 1.5-1.6s for both webpack and utoopack. I did not treat that as conclusive browser bundle readiness data.
I also tried a broader transformer-level optimization that reused dynamic imports and enhanced-resolve setup, but the 120-page benchmark did not improve and it exposed a race in the current
rehypeRawasync ESM preloading pattern, so that experiment is intentionally not included in this PR.Validation
pnpm exec eslint src/loaders/markdown/index.tspnpm exec father buildpnpm test