build: make build faster (at least on Windows)

[refack]

• Version: *
• Platform: Windows
• Subsystem: build

The problem: even for minor changes building on Windows can take 10s of minutes.
This issue is for tracking several approaches to mitigate that.

• make GYP less aggressive in recreating *.*proj files
• better signaling to VS of the minimal change set
• use compiler cache
• make using ninja simpler
• allow for debug build to use /lib/ files as is (skip mksnapshot)
• shortcut the ICU toolchain
• allow decoupling of /deps/ from main executable compilation & linking
• easy way to cloud-source parts of the compilation

[refack]

/cc @nodejs/build @nodejs/platform-windows

[gibfahn]

vcbuild.bat could definitely do with some love, thanks for picking this up @refack!

[refack]

🙏may the powers that be protect us from the evil that is cmd scripting

[refack]

vcbuild.bat could definitely do with some love

Was thinking of rewriting it PowerShell

[gibfahn]

Was thinking of rewriting it PowerShell

Hmm, not sure what the rest of @nodejs/build and @nodejs/platform-windows would think of that, but it sounds like it might be a good idea to me. Fundamentally I think people would be more willing to learn and contribute to a Powershell build script than a batch one, and the tangled mess of gotos in vcbuild.bat doesn't seem particularly contributor-friendly.

Or maybe we should write it in js...

[refack]

Actually could be js more specifically JScript. Windows does have a built in JScript runner

[refack]

Timing example, after applying #12511 (a change in /lib/path.js)

C:\code\node>time < nul&vcbuild > nul&time < nul
The current time is: 14:07:53.76
Enter the new time: The current time is: 14:23:41.10
Enter the new time:

~16 minutes

[sam-github]

Is that faster or slower than the before time?

[refack]

Just an example of a recompile after just changing a .js file :(

Mean while on "Ubuntu on Windows"

refael@refaelux:/mnt/d/code/node$ time BUILDTYPE=Debug make -C out -j3
make: Entering directory '/mnt/d/code/node/out'
  ACTION Regenerating Makefile
-fmake --ignore-environment --toplevel-dir=. -I/mnt/d/code/node/common.gypi -I/mnt/d/code/node/config.gypi --depth=. -Goutput_dir=/mnt/d/code/node/out --generator-output=/mnt/d/code/node/out -Dcomponent=static_library -Dlibrary=static_library -Dlinux_use_bundled_binutils=0 -Dlinux_use_bundled_gold=0 -Dlinux_use_gold_flags=0 node.gyp /mnt/d/code/node/node.gyp -I /mnt/d/code/node/common.gypi -I /mnt/d/code/node/config.gypi --depth=. --generator-output /mnt/d/code/node/out -Goutput_dir=/mnt/d/code/node/out -Dcomponent=static_library -Dlibrary=static_library -Dlinux_use_bundled_binutils=0 -Dlinux_use_bundled_gold=0 -Dlinux_use_gold_flags=0
  ACTION _mnt_d_code_node_node_gyp_node_js2c_host_node_js2c /mnt/d/code/node/out/Debug/obj/gen/node_javascript.cc
  TOUCH 77ac9600461e1dfdf3594f5e11b3cd135d0cbfd6.intermediate
  ACTION Generating inspector protocol sources from protocol json 77ac9600461e1dfdf3594f5e11b3cd135d0cbfd6.intermediate
  CXX(target) /mnt/d/code/node/out/Debug/obj.target/node/gen/node_javascript.o
  LINK(target) /mnt/d/code/node/out/Debug/node
rm 77ac9600461e1dfdf3594f5e11b3cd135d0cbfd6.intermediate
make: Leaving directory '/mnt/d/code/node/out'

real    1m25.091s
user    0m45.625s
sys     0m33.406s

[linux envy]

[sam-github]

Not that much faster for me on Linux. C++ linking of Debug symbols is brutal. I worked with a program that had a 45 minute non-debug linktime once. Debug couldn't be linked, not enough memory on our machines.