Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: allow CJS source map cache to be reclaimed #51711

Merged
merged 1 commit into from
May 28, 2024

Conversation

legendecas
Copy link
Member

Unifies the CJS and ESM source map cache map and allows the CJS cache
entries to be queried more efficiently with a source url without
iteration on an IterableWeakMap. Reclaims the CJS cache entry with
a FinalizationRegistry instead.

Add a regression test to verify that the CJS source map cache entry can be
reclaimed.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Feb 9, 2024
@legendecas legendecas added the source maps Issues and PRs related to source map support. label Feb 9, 2024
@legendecas
Copy link
Member Author

@aduh95 @joyeecheung would you mind taking a look again? Thank you!

run();

// Run until the source map is cleared by GC, or fail the test after determined iterations.
common.gcUntil('SourceMap of deleted CJS module is cleared', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this has the potential of flakes because the strategy used by common.gcUntil() isn't very sound (i.e. sometimes gc() is not aggressive enough to purge things). It would be safer to do it for a bunch more modules (say 30), and checks if any of them can be collected, which reduces the likelihood of the one single module that we are looking at just cannot be garbage collected in time because V8 just doesn't feel pressured enough to do that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, updated the test with more GC pressure to reduce the chance of flaky.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should launch a stress-test CI to make sure we're not introducing any flakiness

lib/internal/source_map/source_map_cache.js Outdated Show resolved Hide resolved
lib/internal/source_map/source_map_cache.js Outdated Show resolved Hide resolved
lib/internal/modules/esm/translators.js Outdated Show resolved Hide resolved
@legendecas legendecas force-pushed the source-maps/cjs branch 2 times, most recently from 9482198 to 611d7db Compare March 18, 2024 11:02
@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 18, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 18, 2024
@nodejs-github-bot
Copy link
Collaborator

@legendecas legendecas requested a review from aduh95 March 19, 2024 03:31
@legendecas
Copy link
Member Author

@joyeecheung would you mind taking another look at this? Thank you!

Copy link
Member

@joyeecheung joyeecheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but this needs a rebase

Unifies the CJS and ESM source map cache map with SourceMapCacheMap
and allows the CJS cache entries to be queried more efficiently with
a source url without iteration on an IterableWeakMap.

Add a test to verify that the CJS source map cache entry can be
reclaimed.
@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label May 27, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 27, 2024
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@legendecas
Copy link
Member Author

CI is green. @joyeecheung @aduh95 would be great to have another review, thanks!

run();

// Run until the source map is cleared by GC, or fail the test after determined iterations.
common.gcUntil('SourceMap of deleted CJS module is cleared', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should launch a stress-test CI to make sure we're not introducing any flakiness

@legendecas
Copy link
Member Author

legendecas commented May 28, 2024

Stress test: https://ci.nodejs.org/job/node-stress-single-test/508 (passed)

@legendecas legendecas added the commit-queue Add this label to land a pull request using GitHub Actions. label May 28, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 28, 2024
@nodejs-github-bot nodejs-github-bot merged commit c0c598d into nodejs:main May 28, 2024
56 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in c0c598d

@legendecas legendecas deleted the source-maps/cjs branch May 28, 2024 23:01
targos pushed a commit that referenced this pull request Jun 1, 2024
Unifies the CJS and ESM source map cache map with SourceMapCacheMap
and allows the CJS cache entries to be queried more efficiently with
a source url without iteration on an IterableWeakMap.

Add a test to verify that the CJS source map cache entry can be
reclaimed.

PR-URL: #51711
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@RafaelGSS RafaelGSS mentioned this pull request Jun 7, 2024
eliphazbouye pushed a commit to eliphazbouye/node that referenced this pull request Jun 20, 2024
Unifies the CJS and ESM source map cache map with SourceMapCacheMap
and allows the CJS cache entries to be queried more efficiently with
a source url without iteration on an IterableWeakMap.

Add a test to verify that the CJS source map cache entry can be
reclaimed.

PR-URL: nodejs#51711
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
bmeck pushed a commit to bmeck/node that referenced this pull request Jun 22, 2024
Unifies the CJS and ESM source map cache map with SourceMapCacheMap
and allows the CJS cache entries to be queried more efficiently with
a source url without iteration on an IterableWeakMap.

Add a test to verify that the CJS source map cache entry can be
reclaimed.

PR-URL: nodejs#51711
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@targos targos added the dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. label Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. needs-ci PRs that need a full CI run. source maps Issues and PRs related to source map support.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants