Repository navigation
Provide a way to load modules dynamically when npm starts without having to specify NODE_OPTIONS #8048
Description
Activity
I think the design as-is solves the problems it solves in the correct way. You either want
NODE_OPTIONSset at a given context, to be set outside of npm, or you want it set for all the things npm calls.I don't know where you got the idea that
npm testwouldn't pick up the required module in this case.~/D/s/scripts $ cat x.js console.log('x is loaded') ~/D/s/scripts $ npm pkg get scripts.s "semver 1.1.1" ~/D/s/scripts $ npm pkg get scripts.f "npm run s" ~/D/s/scripts $ npm run s --node-options='-r ./x.js' > scripts@1.0.0 s > semver 1.1.1 x is loaded 1.1.1 ~/D/s/scripts $ npm pkg get scripts.f "npm run s" ~/D/s/scripts $ npm run f --node-options='-r ./x.js' > scripts@1.0.0 f > npm run s x is loaded > scripts@1.0.0 s > semver 1.1.1 x is loaded 1.1.1
npm runalways spawns a new node process, and it will have whatever--node-optionsyou specify.If you need the initial instance of npm itself to have node-options, you must set it in the environment beforehand. That's a classic bootstrapping problem there.
If you need the initial instance of npm itself to have node-options, you must set it in the environment beforehand. That's a classic bootstrapping problem there.
This is exactly the problem we are having. In order to instrument npm itself, we need to ensure that
NODE_OPTIONSis in the environment beforehand. However, asking users to always runNODE_OPTIONS="-r @gradle/develocity-agent/preload" npm testwould make it quite cumbersome. SettingNODE_OPTIONSglobally might interfere on projects where the module is not installed. We could ask to use a separate tool to manage environment variables, but ideally I would like this to be something that can be achieved without additional tools to avoid unnecessary complexity (and I can imagine there are scenarios where users are not necessarily free to install tooling locally).I'm wondering whether a possible solution could be (and I can imagine this being an opt-in) to add a configuration key to enable this behavior (e.g.
node-options-scope, defaults tolifecycle-scripts-only, could be set tofull) and if enabled, npm forks itself withNODE_OPTIONSset from the configuration. We wouldn't still get access to the original process, but effectively this would allow to setNODE_OPTIONSfrom the config as if it was passed by setting the environment ahead of time.I'm not hard-set neither on this solution or anything similar, I'm trying to figure out what tool we can use to reliably set preload modules when npm is run. I'm wondering if perhaps npm allows to list modules to be loaded in a way similar to how Node.js does with
-r/--require. That would also solve our problem.We wouldn't still get access to the original process.
This is already what's happening if you set
node-optionsin npm config and runnpm test.What part of the original invocation of npm do you need, that you wouldn't need if npm were to re-invoke itself with node-options set? The delta between those two is not a lot of code.
Any solution that exists in npm itself is going to be adding complexity in places it by all rights shouldn't be. I understand the desire to have this managed invisibly to the user, but unfortunately setting the initial environment that runs when you invoke node is (and should remain) outside the scope of npm itself.
I know other tools and languages have scripts that put you into a shell mode where the environment is set as you need. This still seems like the right avenue. This is not complexity we need to add to npm.
What part of the original invocation of npm do you need, that you wouldn't need if npm were to re-invoke itself with node-options set? The delta between those two is not a lot of code.
Not too much indeed, although one thing that we need to track are command line arguments, which IIUC would be lost if
NODE_OPTIONSis not available to the original npm invocation. But our problem is that we need to know that the current process is npm so that we can start instrumenting it. We then instrument any child process spawned by npm itself by propagating context through environment variables, but unless the process that first loads the module is npm, the instrumentation process doesn't start.I know other tools and languages have scripts that put you into a shell mode where the environment is set as you need. This still seems like the right avenue. This is not complexity we need to add to npm.
Absolutely, I don't want to make npm a generic tool in this regard, but identifying a way in which a module can be loaded by npm itself when starting up. Do you think there's some other way to achieve this? Or alternatively, do you think there's some other way of instrumenting npm?
I gave this some additional thought and considered the following alternative: right now, for our use case, we use
NODE_OPTIONSand check if the current process is npm. I considered the alternative of, instead, relying onnode-optionsin the.npmrcand, from the lifecycle script, check if in the chain of parents of the current process, there's one that is npm. Unfortunately I realized that there's a fundamental flaw in this, as the lifecycle script might be some tool that is written in a different language, and would therefore be missed by our instrumentation.Since setting
NODE_OPTIONSgets stuck on a bootstrapping problem, andnode-optionsin the.npmrcmight cause us to miss specific edge cases (which I believe we'd like to make sure we cover), I'm wondering whether there's another way of having the effect of allowing a user configure a module to be loaded dynamically by npm itself before it starts.- changed the title
[-]Allow using npm configuration to set `NODE_OPTIONS` beyond lifecycle scripts[/-][+]Load modules dynamically when npm starts[/+]on Feb 14, 2025 - changed the title
[-]Load modules dynamically when npm starts[/-][+]Provide a way to load modules dynamically when npm starts without having to specify `NODE_OPTIONS`[/+]on Feb 14, 2025 Wrapper process that dynamically sets NODE_OPTIONS then spawns npm with the extra flags?
As of npm 11, it's possible to use the
node-optionsconfiguration key to specify the content ofNODE_OPTIONSfor lifecycle scripts (i.e. it wouldn't be picked up, for example, bynpm test ...-- but it would be fromnpm pretest ...).I'm wondering whether it would be feasible thinking about the idea of allowing users to opt-in into using this configuration key to set
NODE_OPTIONSfor npm commands as well.I'm not hard-set on this solution, I'll share some context around why I'm making this proposal.
Context
As a software engineer at Gradle, I'm part of a team which is adding support for npm to our build observability product, Develocity.
We created a module that can be preloaded (
NODE_OPTIONS="-r ...") and, when enabled, collects events from the run and sends them to the Develocity server.We realized that setting
NODE_OPTIONScan have ergonomics issues: setting it globally means that the module cannot be found for projects that don't have the package installed (and would be picked up by any Node.js process), and unfortunately usingnode-optionsfrom the npm config would not have an effect when running, for examplenpm test.I'm open to discuss other solutions that would fit well with our use case of instrumenting npm.