[BUG] `npx node-gyp@version` runs the copy of node-gyp bundled with npm instead

[DeeDeeG]

Current Behavior:

• npx node-gyp@8 runs the older node-gyp 7.1.2 that is bundled with npm.
  • /home/[me]/n-prefix/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js

Same result if doing plain npx node-gyp. The latest version is v8.0.0, but v7.1.2 is used instead. Happens even if node-gyp v8 is installed globally, and no node-gyp is installed in the location npx is run from.

Expected Behavior:

A separate npx-specific copy of node-gyp v8 should run. Or when node-gyp v8 is installed globally, that could be used.

Steps To Reproduce:

npx node-gyp or npx node-gyp@8 or npx node-gyp@8.0.0, etc.

Environment:

• OS: Ubuntu 20.04, macOS 10.15.7 Catalina, Windows 10
• Node: v14.16.1, v16.0.0-nightly2021041040ace47396
• npm: 7.9.0

[nlf]

i can confirm this, and i see the reason it's happening, but i need to do a bit of research to see why the code is the way it is before we can consider changing it. i'll do some digging and follow up here when i have an answer

[DeeDeeG]

I filed another issue that seems related or at least tangentially related: #2839. I suppose they might be solved at the same time, depending on what the exact details turn out to be.

[DeeDeeG]

I think this started with npm/run-script@e4c7cb3.

As of that commit, @npmcli/run-script adds its bundled node-gyp executable/shim to the PATH. That shim is added relatively early in the PATH (earlier than the dir with npx's downloaded modules). And then @npmcli/run-script runs sh -c node-gyp [args].

So I think the wrong node-gyp is run via what's first in the PATH. (It is always @npmcli/run-script's shim.)

When debugging the execution with node --inspect-brk /home/[me]/npm/cli/bin/npx-cli.js I end up with a PATH var like this:

/home/[me]/[project I'm working in]/node_modules/.bin:/home/[me]/node_modules/.bin:/home/node_modules/.bin:/node_modules/.bin:/home/[me]/npm/cli/node_modules/@npmcli/run-script/lib/node-gyp-bin:/home/[me]/.npm/_npx/[lots o' digits]/node_modules/.bin:[ . . . rest of PATH is my usual PATH content and is not npm-related . . . ]

Note these two:

• /home/[me]/npm/cli/node_modules/@npmcli/run-script/lib/node-gyp-bin
• home/[me]/.npm/_npx/[lots o' digits]/node_modules/.bin

First we have @npmcli/run-script's dir with a node-gyp shim in it, just before home/[me]/.npm/_npx/[lots o' digits]/node_modules/.bin where the npx modules are stored, including the npx copy of node-gyp.

So I do think this has a similar cause to #2839.