fix(pnp): cache the manifest by its own path

[elbywan]

Goal

In a nutshell: this PR aims to fix a huge memory consumption issue when using the pnp feature in a large codebase.

As of today the Plug'n'Play manifest is read/deserialized from the.pnp.cjs file into a struct which is then cached by the path of the file containing the import/require statement. Is seems inefficient since the path can change a lot for large codebases even though the manifest contents are ultimately the same.

This PR changes the code to use the path of the manifest itself as the key to avoid storing the same manifest multiple times.

Context

In my current company we are in the process of migrating from esbuild to rspack to build our frontend assets in a huge monorepo (>10 millions LOC). We are also experimenting with Module Federation.

Our rspack build pipeline works well except when enabling the Module Federation plugin, memory consumption goes through the roof quickly reaching 128+Gb on my machine.

I noticed that one of the main difference between these runs is that we are using our own custom resolver plugin based on enhanced-resolve whereas Module Federation heavily uses rspack-resolver like here.

In order to pinpoint the issue I used the instruments osx app to profile the build and monitor the allocations, and here are some results that prompted me to make these changes:

Heavy memory consumption

---

The load_pnp_manifest() method is called in a redundant way

---

With this fix: no more memory leak

[CLAassistant]

All committers have signed the CLA.

[arcanis]

did you try caching by cached_path.to_path_bug().parent()? All files from a same folder should have the same PnP manifest path (and find_closest_pnp_manifest_path accepts folders).

[elbywan]

I'll check this out 👍

[elbywan]

It seems like using the parent makes these tests fail for some reason: 😬

---- tests::pnp::pnp1 stdout ----
thread 'tests::pnp::pnp1' panicked at src/tests/pnp.rs:19:5:
assertion `left == right` failed
  left: Err(NotFound("is-even"))
 right: Ok("/Users/julien.elbaz/dev/experiments/rspack-resolver/fixtures/pnp/.yarn/cache/is-even-npm-1.0.0-9f726520dc-2728cc2f39.zip/node_modules/is-even/index.js")

---- tests::pnp::resolve_in_pnp_linked_folder stdout ----
thread 'tests::pnp::resolve_in_pnp_linked_folder' panicked at src/tests/pnp.rs:106:5:
assertion `left == right` failed
  left: Err(NotFound("lib/lib.js"))
 right: Ok("/Users/julien.elbaz/dev/experiments/rspack-resolver/fixtures/pnp/shared/lib.js")

I'll leave this optimization out for the moment.

[codspeed-hq]

CodSpeed Performance Report

Merging #69 will improve performances by ×5.2

_{Comparing elbywan:fix-pnp-manifest-caching (75ba6fb) with main (5974ee3)}

Summary

⚡ 1 improvements
✅ 2 untouched benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
⚡	resolver[pnp resolve]	11.3 ms	2.2 ms	×5.2

[stormslowly]

hi @elbywan thank you for your great work!
coudl you plz add codspeed benchmark to prevent the performance downgrading.

Later, will move the pnp functions to CachedPath for more consistent coding style.

[stormslowly]

CodSpeed Performance Report

Merging #69 will improve performances by ×5.2

Comparing elbywan:fix-pnp-manifest-caching (75ba6fb) with main (5974ee3)

Summary

⚡ 1 improvements ✅ 2 untouched benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
⚡ resolver[pnp resolve] 11.3 ms 2.2 ms ×5.2

nice job!