vs2019 failing

[RafaelGSS]

Hey, I've been running the CI and vs2019 keeps failing without apparent error:

Examples:

• https://ci.nodejs.org/job/node-compile-windows/51805/nodes=win-vs2019/console
• https://ci.nodejs.org/job/node-compile-windows/51804/nodes=win-vs2019/console

[targos]

@StefanStojanovic

[richardlau]

Examples:

• https://ci.nodejs.org/job/node-compile-windows/51805/nodes=win-vs2019/console
• https://ci.nodejs.org/job/node-compile-windows/51804/nodes=win-vs2019/console

These failed on test-rackspace-win2019_vs2019-x64-3 -- that machine (or any test-rackspace-win2019* machine) isn't in our inventory (yet?):

build/ansible/inventory.yml

Lines 316 to 330 in f7e0f90

	- rackspace:
	centos7-x64-1: {ip: 119.9.27.82}
	debian8-x64-1: {ip: 23.253.109.216}
	debian10-x64-1: {ip: 104.239.140.184}
	fedora32-x64-1: {ip: 119.9.51.79}
	ubuntu1604-x64-1: {ip: 119.9.51.176}
	ubuntu1604-x64-2: {ip: 104.130.124.194}
	win2012r2_vs2015-x64-1: {ip: 104.239.174.8}
	win2012r2_vs2015-x64-2: {ip: 104.130.141.137}
	win2012r2_vs2019-x64-1: {ip: 162.242.237.124}
	win2012r2_vs2019-x64-2: {ip: 104.130.158.58}
	win2012r2_vs2019-x64-3: {ip: 119.9.131.63}
	win2012r2_vs2019-x64-4: {ip: 104.130.219.103}
	win2012r2_vs2019-x64-6: {ip: 104.130.141.231}

I assume the win2019* machines are part of the work going on under #3373

Looking at the machines with win-vs2019 label it looks like there are still two win2012r2-vs2019 machines available, so we could try temporarily disabling test-rackspace-win2019_vs2019-x64-3 and/or the other win2019-vs2019 machines to force the builds onto the win2012r2 ones.

[richardlau]

Looking at the machines with win-vs2019 label it looks like there are still two win2012r2-vs2019 machines available, so we could try temporarily disabling test-rackspace-win2019_vs2019-x64-3 and/or the other win2019-vs2019 machines to force the builds onto the win2012r2 ones.

In the interests of not blocking the security release, I've marked all the win2019-vs2019 machines offline in Jenkins to force builds onto the win2012r2-vs2019 machines.

cc @StefanStojanovic

[richardlau]

It looks like win2019-vs2019 is needed for Node.js 20 builds, so I reenabled -1 and -2 (-3 and -4 remain offline as they were the ones erroring).

[richardlau]

Examples:

* https://ci.nodejs.org/job/node-compile-windows/51805/nodes=win-vs2019/console

* https://ci.nodejs.org/job/node-compile-windows/51804/nodes=win-vs2019/console

These are Node.js 18 and 16 builds. Here are more failures on win2019-vs2019 for Node.js 16.x:

• https://ci.nodejs.org/job/node-compile-windows/51831/nodes=win-vs2019/console
• https://ci.nodejs.org/job/node-compile-windows/51870/nodes=win-vs2019/console

It looks like the win2019 VMs do not have WiX 3 installed:
e.g. https://ci.nodejs.org/job/node-compile-windows/51831/nodes=win-vs2019/console

11:03:32 > if defined msi (
11:03:32 echo Looking for WiX installation for Visual Studio 2019...  
11:03:32  if not exist "C:\Program Files (x86)\WiX Toolset v3.11\\SDK\VS2017" (
11:03:32 echo Failed to find WiX install for Visual Studio 2019  
11:03:32  echo VS2019 support for WiX is only present starting at version 3.11  
11:03:32  goto msbuild-not-found 
11:03:32 )  
11:03:32  if not exist "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\\..\MSBuild\Microsoft\WiX" (
11:03:32 echo Failed to find the WiX Toolset Visual Studio 2019 Extension  
11:03:32  goto msbuild-not-found 
11:03:32 ) 
11:03:32 ) 
11:03:32 Looking for WiX installation for Visual Studio 2019...
11:03:32 Failed to find the WiX Toolset Visual Studio 2019 Extension
11:03:32 
11:03:32 > echo Failed to find a suitable Visual Studio installation. 
11:03:32 Failed to find a suitable Visual Studio installation.
11:03:32 
11:03:32 > echo Try to run in a "Developer Command Prompt" or consult 
11:03:32 Try to run in a "Developer Command Prompt" or consult
11:03:32 
11:03:32 > echo https://github.com/nodejs/node/blob/HEAD/BUILDING.md#windows 
11:03:32 https://github.com/nodejs/node/blob/HEAD/BUILDING.md#windows
11:03:32 

There's a further weird looking error but I can't tell if it's a separate problem or a follow on from WiX not being found causing things to not be built:

11:03:32 > tar cavf binary/binary.tar.gz config.gypi icu_config.gypi Release/node.exe Release/node.lib Release/openssl-cli.exe Release/cctest.exe Release/node.pdb 
11:03:32 tar: hocolatey\bin;C:\Pro???? Files\Git\cmd;C:\Pr????m Files\Git\mingw64\????C:\Program Files\Git????\bin;C:\Program File??????????????: Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: : Couldn't visit directory: No such file or directory
11:03:32 tar: Error exit delayed from previous errors.
11:03:32 

The migration to WiX 4 (nodejs/node#45943) (semver-minor) hasn't been backported to Node.js 18 or 16. I'm hesitant to backport to Node.js 16 this close to its End-of-Life -- thoughts @nodejs/lts?

Anyway I've reenabled all the win2019-vs2019 VMs in Jenkins -- I believe that they are fine for builds for Node.js 20 and main and having more VMs online should help the current Jenkins backlog.

[richardlau]

Made a temporary change to the agent config for the win2019 VMs -- I've removed the labels

win-vs2019 win-vs2019-x86 win-vs2019-arm64

so the node-compile-windows CI job runs on the win2012r2-vs2019 VMs. If we can get WiX 3 set up on the win2109 VMs we can put those labels back.

cc @StefanStojanovic @joaocgreis

The migration to WiX 4 (nodejs/node#45943) (semver-minor) hasn't been backported to Node.js 18 or 16. I'm hesitant to backport to Node.js 16 this close to its End-of-Life -- thoughts https://github.com/orgs/nodejs/teams/lts?

FWIW I tried a quick cherry-pick of nodejs/node#45943 locally onto v18.x-staging and v16.x-staging and there were merge conflicts (both branches) which reinforces my hesitancy to backport that to v16.x this close to Node.js 16 End-of-Life next month.

[StefanStojanovic]

Hello everyone, sorry for replying just now. I was on vacation and I'm currently not working full-time on Node.js. The issue was not WiX 3, but rather its VS extension which was missing on some machines, although an ansible playbook should install them. Anyway, I fixed that and added 2 machines back to the compilation, the plan is to add the others in a few days too. Also, I made a PR updating inventory with the new machines #3460.

[StefanStojanovic]

Since this issue was fixed manually I didn't see it repeating afterward. I also made an issue for myself to fix it in Ansible. With all this in mind, I'll close the issue and if needed it can be reopened.