Skip to content

chore: try reuse factorize module#13344

Open
hardfist wants to merge 1 commit intomainfrom
codex/factorizetask-visited-cachedashmapfactorize
Open

chore: try reuse factorize module#13344
hardfist wants to merge 1 commit intomainfrom
codex/factorizetask-visited-cachedashmapfactorize

Conversation

@hardfist
Copy link
Contributor

No description provided.

Copilot AI review requested due to automatic review settings March 13, 2026 10:01
@github-actions github-actions bot added the team The issue/pr is created by the member of Rspack. label Mar 13, 2026
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6975c87f6f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +129 to +130
if is_normal_module {
tasks.extend(take_normal_module_dedupe_tasks(context, module_identifier));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Release dedupe waiters when module ceases to be NormalModule

This only wakes dedupe waiters when the added module is still a NormalModule, but the claim is taken earlier in normal_module_factory before normal_module_factory_hooks.module can replace it. Hooks such as DelegatedPlugin::nmf_module can swap module to DelegatedModule, so the owner task never calls mark_ready and all deduped requests remain stuck behind PendingNormalModuleDedupeTask, leaving those dependencies unresolved in the module graph.

Useful? React with 👍 / 👎.

Comment on lines +149 to +150
.normal_module_dedupe_tracker
.take_waiters_and_clear(module_identifier)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid clearing dedupe state before waiters can register

On a factorization failure for a claimed normal-module id, this clears the tracker entry in the background task, but waiters are only registered later in FactorizeResultTask::main_run. If a deduped sibling finishes after this clear, register_waiter_or_get_ready sees a vacant entry and treats it as ready, causing ReuseNormalModuleTask to resolve dependencies to a module identifier that was never added.

Useful? React with 👍 / 👎.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 13, 2026

📝 Benchmark detail: Open

No benchmark data for commit 93af830, trying parent...
No benchmark data for commit 3e83955, trying parent...

Name Base (main cf2c8bc) Current Change
10000_big_production-mode_disable-minimize + exec 14.3 s ± 368 ms 14.3 s ± 272 ms -0.14 %
10000_development-mode + exec 962 ms ± 17 ms 929 ms ± 27 ms -3.45 %
10000_development-mode_hmr + stats 176 ms ± 5.7 ms 174 ms ± 4.9 ms -1.14 %
10000_development-mode_noop-loader + exec 2.08 s ± 39 ms 2.08 s ± 38 ms -0.35 %
10000_production-mode + exec 1.12 s ± 17 ms 1.08 s ± 14 ms -2.83 %
10000_production-mode_persistent-cold + exec 1.29 s ± 32 ms 1.28 s ± 32 ms -1.36 %
10000_production-mode_persistent-hot + exec 988 ms ± 27 ms 977 ms ± 14 ms -1.17 %
10000_production-mode_source-map + exec 1.28 s ± 27 ms 1.27 s ± 23 ms -0.76 %
arco-pro_development-mode + exec 1.34 s ± 78 ms 1.35 s ± 41 ms +0.48 %
arco-pro_development-mode_hmr + stats 37 ms ± 0.66 ms 38 ms ± 4.1 ms +2.94 %
arco-pro_production-mode + exec 2.47 s ± 75 ms 2.47 s ± 47 ms -0.08 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 2.52 s ± 99 ms 2.52 s ± 79 ms -0.29 %
arco-pro_production-mode_persistent-cold + exec 2.53 s ± 41 ms 2.54 s ± 46 ms +0.40 %
arco-pro_production-mode_persistent-hot + exec 1.4 s ± 63 ms 1.39 s ± 37 ms -0.43 %
arco-pro_production-mode_source-map + exec 2.91 s ± 76 ms 2.93 s ± 123 ms +0.60 %
arco-pro_production-mode_traverse-chunk-modules + exec 2.49 s ± 90 ms 2.51 s ± 44 ms +0.70 %
bundled-threejs_development-mode + exec 195 ms ± 3.5 ms 194 ms ± 3.5 ms -0.51 %
bundled-threejs_production-mode + exec 229 ms ± 5.6 ms 227 ms ± 4.8 ms -1.13 %
large-dyn-imports_development-mode + exec 1.2 s ± 13 ms 1.2 s ± 40 ms +0.25 %
large-dyn-imports_production-mode + exec 1.32 s ± 34 ms 1.34 s ± 43 ms +1.61 %
threejs_development-mode_10x + exec 856 ms ± 37 ms 841 ms ± 11 ms -1.73 %
threejs_development-mode_10x_hmr + stats 119 ms ± 2.8 ms 118 ms ± 3.7 ms -1.02 %
threejs_production-mode_10x + exec 3.09 s ± 45 ms 3.08 s ± 32 ms -0.36 %
threejs_production-mode_10x_persistent-cold + exec 3.22 s ± 35 ms 3.21 s ± 30 ms -0.14 %
threejs_production-mode_10x_persistent-hot + exec 2.62 s ± 26 ms 2.63 s ± 25 ms +0.21 %
threejs_production-mode_10x_source-map + exec 3.82 s ± 86 ms 3.81 s ± 98 ms -0.09 %
10000_big_production-mode_disable-minimize + rss memory 2017 MiB ± 77.8 MiB 2029 MiB ± 51.6 MiB +0.60 %
10000_development-mode + rss memory 595 MiB ± 9.82 MiB 592 MiB ± 11.3 MiB -0.45 %
10000_development-mode_hmr + rss memory 810 MiB ± 39.4 MiB 814 MiB ± 34.8 MiB +0.48 %
10000_development-mode_noop-loader + rss memory 893 MiB ± 10.7 MiB 888 MiB ± 7.61 MiB -0.62 %
10000_production-mode + rss memory 546 MiB ± 21.4 MiB 537 MiB ± 12.1 MiB -1.68 %
10000_production-mode_persistent-cold + rss memory 737 MiB ± 19.1 MiB 735 MiB ± 1.74 MiB -0.33 %
10000_production-mode_persistent-hot + rss memory 745 MiB ± 16.2 MiB 747 MiB ± 17.7 MiB +0.33 %
10000_production-mode_source-map + rss memory 564 MiB ± 5.86 MiB 568 MiB ± 12.2 MiB +0.77 %
arco-pro_development-mode + rss memory 475 MiB ± 7.54 MiB 475 MiB ± 9.39 MiB +0.03 %
arco-pro_development-mode_hmr + rss memory 492 MiB ± 15.8 MiB 496 MiB ± 11.4 MiB +0.74 %
arco-pro_production-mode + rss memory 661 MiB ± 59.5 MiB 663 MiB ± 14.3 MiB +0.24 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 689 MiB ± 14.1 MiB 676 MiB ± 68.4 MiB -1.96 %
arco-pro_production-mode_persistent-cold + rss memory 749 MiB ± 8.7 MiB 747 MiB ± 15.5 MiB -0.17 %
arco-pro_production-mode_persistent-hot + rss memory 568 MiB ± 32.6 MiB 553 MiB ± 14.8 MiB -2.51 %
arco-pro_production-mode_source-map + rss memory 764 MiB ± 15 MiB 750 MiB ± 23.2 MiB -1.94 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 673 MiB ± 17.4 MiB 672 MiB ± 8.11 MiB -0.15 %
bundled-threejs_development-mode + rss memory 161 MiB ± 5.66 MiB 158 MiB ± 6.73 MiB -2.11 %
bundled-threejs_production-mode + rss memory 180 MiB ± 9.77 MiB 181 MiB ± 14.7 MiB +0.33 %
large-dyn-imports_development-mode + rss memory 602 MiB ± 30.3 MiB 589 MiB ± 45.1 MiB -2.27 %
large-dyn-imports_production-mode + rss memory 455 MiB ± 5.4 MiB 456 MiB ± 9.15 MiB +0.35 %
threejs_development-mode_10x + rss memory 538 MiB ± 24.4 MiB 536 MiB ± 17.6 MiB -0.32 %
threejs_development-mode_10x_hmr + rss memory 670 MiB ± 10.7 MiB 676 MiB ± 29.2 MiB +1.02 %
threejs_production-mode_10x + rss memory 720 MiB ± 14 MiB 710 MiB ± 5.68 MiB -1.35 %
threejs_production-mode_10x_persistent-cold + rss memory 854 MiB ± 16.9 MiB 853 MiB ± 12.4 MiB -0.07 %
threejs_production-mode_10x_persistent-hot + rss memory 701 MiB ± 10.5 MiB 702 MiB ± 10.8 MiB +0.16 %
threejs_production-mode_10x_source-map + rss memory 851 MiB ± 14.4 MiB 859 MiB ± 23.6 MiB +0.93 %

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a NormalModule deduplication mechanism during module graph repair/factorization to avoid creating/factorizing the same NormalModule multiple times and instead reuse an existing/in-progress module by identifier.

Changes:

  • Extend ModuleFactoryResult / ModuleFactoryCreateData to support “deduped normal module” signaling and tracking.
  • Add NormalModuleDedupeTracker and wire it through the repair task pipeline to coordinate reuse.
  • Update NormalModuleFactory to claim identifiers and return a dedupe result when an identical NormalModule is already claimed.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
crates/rspack_plugin_dll/src/dll_entry/dll_module_factory.rs Populate the new normal_module_dedup field in factory result.
crates/rspack_core/src/normal_module_factory.rs Claim NormalModule identifiers and return dedupe-only factory results when already claimed.
crates/rspack_core/src/normal_module.rs Expose create_id as pub(crate) for identifier computation.
crates/rspack_core/src/module_factory.rs Add dedupe tracker + waiter types; extend factory input/output structs to carry dedupe info.
crates/rspack_core/src/context_module_factory.rs Populate the new normal_module_dedup field in factory result.
crates/rspack_core/src/compilation/build_module_graph/module_executor/entry.rs Thread the dedupe tracker into executor-created factorize tasks.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/process_dependencies.rs Thread the dedupe tracker into factorize tasks spawned from dependency processing.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/mod.rs Initialize and propagate the shared dedupe tracker for repair runs.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/context.rs Add dedupe tracker storage to TaskContext.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/factorize.rs Handle dedupe-only factory results by queueing waiters and reusing existing modules.
crates/rspack_core/src/compilation/build_module_graph/graph_updater/repair/add.rs Mark dedupe entries ready when a NormalModule is added and schedule reuse tasks for waiters.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +60 to +63
ready: true,
waiters: vec![],
});
Some(waiter)
Comment on lines +144 to +162
if factorize_failed
&& let Some(module_identifier) = claimed_normal_module_identifier
&& let Some(waiter_failure_info) = waiter_failure_info.as_ref()
{
for waiter in create_data
.normal_module_dedupe_tracker
.take_waiters_and_clear(module_identifier)
{
let factorize_info = clone_factorize_failure_info(waiter_failure_info, &waiter);
tasks.push(Box::new(FactorizeResultTask {
original_module_identifier: waiter.original_module_identifier,
factory_result: None,
normal_module_dedup: None,
dependencies: waiter.dependencies,
factorize_info,
from_unlazy: waiter.from_unlazy,
}));
}
}
Comment on lines +160 to +174
pub normal_module_dedup: Option<ModuleIdentifier>,
}

impl ModuleFactoryResult {
pub fn new_with_module(module: BoxModule) -> Self {
Self {
module: Some(module),
normal_module_dedup: None,
}
}

pub fn new_with_normal_module_dedup(module_identifier: ModuleIdentifier) -> Self {
Self {
module: None,
normal_module_dedup: Some(module_identifier),
@github-actions
Copy link
Contributor

Rsdoctor Bundle Diff Analysis

⚠️ Note: The latest commit (3e839559d3) does not have baseline artifacts. Using commit cf2c8bc4eb for baseline comparison instead. If this seems incorrect, please wait a few minutes and try rerunning the workflow.

Found 5 projects in monorepo, 0 projects with changes.

📊 Quick Summary
Project Total Size Change
react-10k 5.7 MB 0
react-1k 826.2 KB 0
react-5k 2.7 MB 0
rome 984.2 KB 0
ui-components 2.3 MB 0

Generated by Rsdoctor GitHub Action

@github-actions
Copy link
Contributor

📦 Binary Size-limit

Comparing 6975c87 to refactor: remove Ukey DataBase abstraction (#13339) by hardfist

❌ Size increased by 20.88KB from 49.04MB to 49.06MB (⬆️0.04%)

@codspeed-hq
Copy link

codspeed-hq bot commented Mar 13, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks
⏩ 3 skipped benchmarks1


Comparing codex/factorizetask-visited-cachedashmapfactorize (6975c87) with main (cf2c8bc)2

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (3e83955) during the generation of this report, so cf2c8bc was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 13, 2026

📝 Benchmark detail: Open

Name Base (main 93af830) Current Change
10000_big_production-mode_disable-minimize + exec 14.3 s ± 245 ms 14.3 s ± 296 ms -0.17 %
10000_development-mode + exec 955 ms ± 27 ms 933 ms ± 17 ms -2.35 %
10000_development-mode_hmr + stats 175 ms ± 4.1 ms 172 ms ± 2.4 ms -1.32 %
10000_development-mode_noop-loader + exec 2.1 s ± 79 ms 2.07 s ± 81 ms -1.67 %
10000_production-mode + exec 1.1 s ± 7.9 ms 1.08 s ± 27 ms -1.59 %
10000_production-mode_persistent-cold + exec 1.28 s ± 35 ms 1.28 s ± 27 ms -0.41 %
10000_production-mode_persistent-hot + exec 976 ms ± 26 ms 965 ms ± 20 ms -1.11 %
10000_production-mode_source-map + exec 1.28 s ± 24 ms 1.26 s ± 17 ms -1.42 %
arco-pro_development-mode + exec 1.37 s ± 66 ms 1.33 s ± 50 ms -3.41 %
arco-pro_development-mode_hmr + stats 37 ms ± 0.51 ms 37 ms ± 0.75 ms +0.25 %
arco-pro_production-mode + exec 2.48 s ± 111 ms 2.47 s ± 41 ms -0.52 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 2.51 s ± 96 ms 2.5 s ± 66 ms -0.56 %
arco-pro_production-mode_persistent-cold + exec 2.52 s ± 74 ms 2.5 s ± 59 ms -0.53 %
arco-pro_production-mode_persistent-hot + exec 1.38 s ± 43 ms 1.4 s ± 43 ms +1.58 %
arco-pro_production-mode_source-map + exec 2.9 s ± 104 ms 2.93 s ± 93 ms +1.06 %
arco-pro_production-mode_traverse-chunk-modules + exec 2.5 s ± 60 ms 2.48 s ± 88 ms -0.52 %
bundled-threejs_development-mode + exec 193 ms ± 2 ms 194 ms ± 5.1 ms +0.41 %
bundled-threejs_production-mode + exec 228 ms ± 2 ms 229 ms ± 8.2 ms +0.53 %
large-dyn-imports_development-mode + exec 1.19 s ± 13 ms 1.19 s ± 28 ms -0.37 %
large-dyn-imports_production-mode + exec 1.33 s ± 39 ms 1.32 s ± 41 ms -0.57 %
threejs_development-mode_10x + exec 846 ms ± 17 ms 850 ms ± 15 ms +0.50 %
threejs_development-mode_10x_hmr + stats 119 ms ± 7.4 ms 119 ms ± 5.1 ms +0.32 %
threejs_production-mode_10x + exec 3.09 s ± 54 ms 3.11 s ± 38 ms +0.78 %
threejs_production-mode_10x_persistent-cold + exec 3.21 s ± 49 ms 3.2 s ± 29 ms -0.59 %
threejs_production-mode_10x_persistent-hot + exec 2.59 s ± 41 ms 2.61 s ± 13 ms +0.63 %
threejs_production-mode_10x_source-map + exec 3.81 s ± 54 ms 3.79 s ± 61 ms -0.54 %
10000_big_production-mode_disable-minimize + rss memory 2037 MiB ± 45.9 MiB 2024 MiB ± 59.2 MiB -0.66 %
10000_development-mode + rss memory 596 MiB ± 18.3 MiB 595 MiB ± 21.3 MiB -0.23 %
10000_development-mode_hmr + rss memory 809 MiB ± 31.6 MiB 813 MiB ± 35.2 MiB +0.49 %
10000_development-mode_noop-loader + rss memory 894 MiB ± 16.2 MiB 888 MiB ± 10.6 MiB -0.71 %
10000_production-mode + rss memory 545 MiB ± 9.86 MiB 540 MiB ± 12.2 MiB -0.87 %
10000_production-mode_persistent-cold + rss memory 741 MiB ± 14.1 MiB 736 MiB ± 23.9 MiB -0.68 %
10000_production-mode_persistent-hot + rss memory 747 MiB ± 8.43 MiB 747 MiB ± 2.93 MiB -0.01 %
10000_production-mode_source-map + rss memory 564 MiB ± 10.9 MiB 564 MiB ± 7.5 MiB +0.07 %
arco-pro_development-mode + rss memory 475 MiB ± 5.12 MiB 477 MiB ± 5.27 MiB +0.48 %
arco-pro_development-mode_hmr + rss memory 489 MiB ± 14.2 MiB 491 MiB ± 9.42 MiB +0.46 %
arco-pro_production-mode + rss memory 659 MiB ± 49.8 MiB 659 MiB ± 63.8 MiB -0.02 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 690 MiB ± 20.8 MiB 691 MiB ± 9.4 MiB +0.04 %
arco-pro_production-mode_persistent-cold + rss memory 744 MiB ± 19.5 MiB 748 MiB ± 10.2 MiB +0.61 %
arco-pro_production-mode_persistent-hot + rss memory 563 MiB ± 14.1 MiB 557 MiB ± 15.3 MiB -1.08 %
arco-pro_production-mode_source-map + rss memory 755 MiB ± 20.5 MiB 759 MiB ± 28.7 MiB +0.48 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 665 MiB ± 71.2 MiB 670 MiB ± 12.9 MiB +0.85 %
bundled-threejs_development-mode + rss memory 159 MiB ± 11.2 MiB 157 MiB ± 10.2 MiB -0.84 %
bundled-threejs_production-mode + rss memory 180 MiB ± 8.55 MiB 180 MiB ± 4.8 MiB +0.07 %
large-dyn-imports_development-mode + rss memory 594 MiB ± 9.42 MiB 599 MiB ± 20.2 MiB +0.86 %
large-dyn-imports_production-mode + rss memory 454 MiB ± 2.91 MiB 458 MiB ± 8.76 MiB +0.81 %
threejs_development-mode_10x + rss memory 536 MiB ± 17.1 MiB 537 MiB ± 10 MiB +0.16 %
threejs_development-mode_10x_hmr + rss memory 676 MiB ± 16.3 MiB 672 MiB ± 11.4 MiB -0.66 %
threejs_production-mode_10x + rss memory 723 MiB ± 16.8 MiB 717 MiB ± 5.4 MiB -0.78 %
threejs_production-mode_10x_persistent-cold + rss memory 850 MiB ± 7.47 MiB 859 MiB ± 6.8 MiB +1.06 %
threejs_production-mode_10x_persistent-hot + rss memory 697 MiB ± 6.65 MiB 704 MiB ± 9.53 MiB +0.94 %
threejs_production-mode_10x_source-map + rss memory 851 MiB ± 27.2 MiB 859 MiB ± 22 MiB +0.82 %

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team The issue/pr is created by the member of Rspack.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants