Skip to content

perf: Refactor factorize dependency tracking#13306

Draft
hardfist wants to merge 4 commits intomainfrom
codex/change-factorize-task-dependencies
Draft

perf: Refactor factorize dependency tracking#13306
hardfist wants to merge 4 commits intomainfrom
codex/change-factorize-task-dependencies

Conversation

@hardfist
Copy link
Contributor

Summary

  • replace Vec with ModuleDependencies so only the primary dependency is cloned and the ids list stays lightweight
  • propagate the new structure through process_dependencies, factorize, add/overwrite tasks, and use module graph data to compute forwarded IDs without cloning
  • store factorize info only on the primary dependency and keep other dependency factors in sync via ModuleDependencies metadata

Testing

  • Not run (not requested)

Copilot AI review requested due to automatic review settings March 11, 2026 09:33
@github-actions github-actions bot added the team The issue/pr is created by the member of Rspack. label Mar 11, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2026

📝 Benchmark detail: Open

Name Base (2026-03-11 2b754ef) Current Change
10000_big_production-mode_disable-minimize + exec 14.1 s ± 105 ms 14.2 s ± 88 ms +0.54 %
10000_development-mode + exec 1.02 s ± 36 ms 997 ms ± 26 ms -2.22 %
10000_development-mode_hmr + stats 176 ms ± 3.8 ms 175 ms ± 6.9 ms -0.64 %
10000_development-mode_noop-loader + exec 2.07 s ± 57 ms 2.09 s ± 68 ms +0.84 %
10000_production-mode + exec 1.1 s ± 42 ms 1.09 s ± 29 ms -0.57 %
10000_production-mode_persistent-cold + exec 1.28 s ± 30 ms 1.27 s ± 41 ms -0.89 %
10000_production-mode_persistent-hot + exec 985 ms ± 14 ms 975 ms ± 11 ms -1.04 %
10000_production-mode_source-map + exec 1.29 s ± 14 ms 1.27 s ± 16 ms -1.12 %
arco-pro_development-mode + exec 1.33 s ± 60 ms 1.36 s ± 52 ms +1.98 %
arco-pro_development-mode_hmr + stats 37 ms ± 1.5 ms 38 ms ± 5.6 ms +1.56 %
arco-pro_production-mode + exec 2.46 s ± 71 ms 2.5 s ± 82 ms +1.35 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 2.52 s ± 74 ms 2.55 s ± 75 ms +1.32 %
arco-pro_production-mode_persistent-cold + exec 2.51 s ± 121 ms 2.56 s ± 114 ms +2.13 %
arco-pro_production-mode_persistent-hot + exec 1.41 s ± 33 ms 1.41 s ± 45 ms +0.50 %
arco-pro_production-mode_source-map + exec 2.91 s ± 102 ms 2.92 s ± 129 ms +0.10 %
arco-pro_production-mode_traverse-chunk-modules + exec 2.48 s ± 63 ms 2.49 s ± 104 ms +0.34 %
bundled-threejs_development-mode + exec 195 ms ± 5.1 ms 196 ms ± 4.6 ms +0.51 %
bundled-threejs_production-mode + exec 228 ms ± 7.6 ms 229 ms ± 7.9 ms +0.09 %
large-dyn-imports_development-mode + exec 1.27 s ± 24 ms 1.25 s ± 25 ms -1.26 %
large-dyn-imports_production-mode + exec 1.34 s ± 49 ms 1.32 s ± 51 ms -1.42 %
threejs_development-mode_10x + exec 852 ms ± 14 ms 851 ms ± 19 ms -0.12 %
threejs_development-mode_10x_hmr + stats 117 ms ± 1.5 ms 119 ms ± 4.5 ms +1.46 %
threejs_production-mode_10x + exec 3.06 s ± 39 ms 3.1 s ± 53 ms +1.04 %
threejs_production-mode_10x_persistent-cold + exec 3.2 s ± 44 ms 3.21 s ± 47 ms +0.19 %
threejs_production-mode_10x_persistent-hot + exec 2.6 s ± 18 ms 2.62 s ± 37 ms +0.65 %
threejs_production-mode_10x_source-map + exec 3.77 s ± 50 ms 3.79 s ± 41 ms +0.57 %
10000_big_production-mode_disable-minimize + rss memory 2022 MiB ± 53.4 MiB 2045 MiB ± 35 MiB +1.13 %
10000_development-mode + rss memory 664 MiB ± 14.6 MiB 634 MiB ± 8.78 MiB -4.55 %
10000_development-mode_hmr + rss memory 883 MiB ± 35.7 MiB 853 MiB ± 63.8 MiB -3.40 %
10000_development-mode_noop-loader + rss memory 960 MiB ± 18.8 MiB 941 MiB ± 19.3 MiB -2.02 %
10000_production-mode + rss memory 550 MiB ± 9.95 MiB 539 MiB ± 8.57 MiB -2.04 %
10000_production-mode_persistent-cold + rss memory 752 MiB ± 14.4 MiB 739 MiB ± 21.9 MiB -1.74 %
10000_production-mode_persistent-hot + rss memory 755 MiB ± 19.1 MiB 746 MiB ± 6.58 MiB -1.18 %
10000_production-mode_source-map + rss memory 572 MiB ± 14.1 MiB 568 MiB ± 13.9 MiB -0.86 %
arco-pro_development-mode + rss memory 479 MiB ± 8.21 MiB 477 MiB ± 5.29 MiB -0.53 %
arco-pro_development-mode_hmr + rss memory 494 MiB ± 11.5 MiB 494 MiB ± 3.42 MiB +0.08 %
arco-pro_production-mode + rss memory 661 MiB ± 61.2 MiB 673 MiB ± 11 MiB +1.71 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 684 MiB ± 60.3 MiB 676 MiB ± 65.2 MiB -1.13 %
arco-pro_production-mode_persistent-cold + rss memory 747 MiB ± 4.64 MiB 743 MiB ± 18.6 MiB -0.55 %
arco-pro_production-mode_persistent-hot + rss memory 553 MiB ± 47.7 MiB 563 MiB ± 34.6 MiB +1.76 %
arco-pro_production-mode_source-map + rss memory 757 MiB ± 51.5 MiB 765 MiB ± 23.6 MiB +0.97 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 675 MiB ± 8.38 MiB 676 MiB ± 9.1 MiB +0.19 %
bundled-threejs_development-mode + rss memory 159 MiB ± 2.53 MiB 159 MiB ± 8.06 MiB +0.32 %
bundled-threejs_production-mode + rss memory 180 MiB ± 9.71 MiB 177 MiB ± 6 MiB -1.38 %
large-dyn-imports_development-mode + rss memory 656 MiB ± 5 MiB 647 MiB ± 11.4 MiB -1.46 %
large-dyn-imports_production-mode + rss memory 463 MiB ± 11.5 MiB 456 MiB ± 8.43 MiB -1.65 %
threejs_development-mode_10x + rss memory 542 MiB ± 9.97 MiB 539 MiB ± 21.8 MiB -0.67 %
threejs_development-mode_10x_hmr + rss memory 685 MiB ± 18.2 MiB 675 MiB ± 19.8 MiB -1.43 %
threejs_production-mode_10x + rss memory 721 MiB ± 21.1 MiB 720 MiB ± 19 MiB -0.16 %
threejs_production-mode_10x_persistent-cold + rss memory 859 MiB ± 10.1 MiB 857 MiB ± 7.1 MiB -0.24 %
threejs_production-mode_10x_persistent-hot + rss memory 708 MiB ± 10.1 MiB 705 MiB ± 13.4 MiB -0.39 %
threejs_production-mode_10x_source-map + rss memory 866 MiB ± 12.9 MiB 854 MiB ± 31.1 MiB -1.35 %

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10c45d3684

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dependencies: self.dependencies,
issuer: self.issuer,
issuer_identifier: self.original_module_identifier,
dependencies: vec![dependency],

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pass all grouped dependencies into module factory hooks

This refactor now calls the module factory with dependencies: vec![dependency], so only the primary dependency is visible to factory hooks and plugins. That regresses behavior for grouped dependencies: for example, ContextReplacementPlugin iterates data.dependencies to clear critical on each context dependency, and finish_modules later reads diagnostics from each dependency in the module graph, so non-primary duplicates can now keep stale critical diagnostics and emit warnings that were previously suppressed. The grouped dependency set still needs to be propagated (or hook-side mutations applied back to every grouped dependency id), not just the first entry.

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 059baef. process_dependencies still groups by DependencyId, but FactorizeTask now rebuilds and passes the full dependency list into ModuleFactoryCreateData, so hook-side mutations on data.dependencies are preserved and synced back to the module graph.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the build-module-graph “repair” pipeline to treat a group of equivalent dependencies as a single “primary” dependency plus a list of associated DependencyIds, updating task wiring to use the new grouped representation.

Changes:

  • Introduce ModuleDependencies to carry a primary BoxDependency alongside grouped DependencyIds.
  • Update factorize/process/add/overwrite/entry tasks to use ModuleDependencies APIs (primary_id, primary_dependency, etc.).
  • Change dependency grouping in ProcessDependenciesTask to accumulate ids into ModuleDependencies instead of collecting Vec<BoxDependency>.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
crates/rspack_core/src/compilation/build_module_graph/module_executor/overwrite.rs Switch dependency id extraction to ModuleDependencies::primary_id() for tracker callbacks.
crates/rspack_core/src/compilation/build_module_graph/module_executor/entry.rs Construct factorize tasks using ModuleDependencies::from_dependency.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/process_dependencies.rs Group dependencies into ModuleDependencies keyed by resource identifier.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/mod.rs Update repair entry-dependency factorization to use ModuleDependencies.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/factorize.rs Add ModuleDependencies and refactor factorize/factorize-result to track grouped ids.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/add.rs Update add flow to resolve modules for all grouped ids and compute forwarded ids via module graph.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +114 to +134
let request = dependency
.as_module_dependency()
.map(|d| d.request().to_string())
.or_else(|| {
self.dependencies[0]
dependency
.as_context_dependency()
.map(|d| d.request().to_string())
})
.unwrap_or_default();
let (dependency, dependency_ids) = dependencies.into_parts();
// Error and result are not mutually exclusive in webpack module factorization.
// Rspack puts results that need to be shared in both error and ok in [ModuleFactoryCreateData].
let mut create_data = ModuleFactoryCreateData {
compiler_id: self.compiler_id,
compilation_id: self.compilation_id,
resolve_options: self.resolve_options,
options: self.options.clone(),
compiler_id,
compilation_id,
resolve_options,
options: options.clone(),
context,
request,
dependencies: self.dependencies,
issuer: self.issuer,
issuer_identifier: self.original_module_identifier,
dependencies: vec![dependency],
issuer,
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FactorizeTask now passes only a single “primary” dependency into ModuleFactoryCreateData (dependencies: vec![dependency]) and later reconstructs ModuleDependencies by taking only the first element from create_data.dependencies. This changes semantics for grouped dependencies: module/context module factory hooks (e.g. ContextReplacementPlugin’s ContextModuleFactoryBeforeResolve) currently iterate and mutate every dependency in data.dependencies; with this change those mutations will only apply to one dependency, and any additional dependencies inserted/returned by hooks will be dropped. To preserve existing hook behavior, keep passing the full dependency list through module factory/hook execution and then sync all mutated dependencies back into the module graph (while still tracking the grouped DependencyIds).

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 059baef. FactorizeTask no longer drops grouped dependencies before module-factory hooks. The full dependency vector flows through create_data.dependencies again, and FactorizeResultTask/AddTask sync all mutated dependencies back into the module graph.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2026

Rsdoctor Bundle Diff Analysis

Found 5 projects in monorepo, 0 projects with changes.

📊 Quick Summary
Project Total Size Change
react-10k 5.7 MB 0
react-1k 826.2 KB 0
react-5k 2.7 MB 0
rome 984.2 KB 0
ui-components 2.3 MB 0

Generated by Rsdoctor GitHub Action

@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2026

📦 Binary Size-limit

Comparing c3f342a to fix(rstest): prevent the re-exports optimization for mocked modules (#13262) by 9aoy

❌ Size increased by 1.88KB from 49.01MB to 49.01MB (⬆️0.00%)

@hardfist hardfist marked this pull request as draft March 11, 2026 09:49
@hardfist hardfist changed the title Refactor factorize dependency tracking perf: Refactor factorize dependency tracking Mar 11, 2026
@github-actions github-actions bot added the release: performance release: performance related release(mr only) label Mar 11, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 11, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks
⏩ 3 skipped benchmarks1


Comparing codex/change-factorize-task-dependencies (c3f342a) with main (802fca1)

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 12, 2026

Deploying rspack with  Cloudflare Pages  Cloudflare Pages

Latest commit: e7ab482
Status: ✅  Deploy successful!
Preview URL: https://85ba7705.rspack-v2.pages.dev
Branch Preview URL: https://codex-change-factorize-task.rspack-v2.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release: performance release: performance related release(mr only) team The issue/pr is created by the member of Rspack.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants