node-test-commit is broken

[aduh95]

It looks like all jobs are failing is less than a second, starting with https://ci.nodejs.org/job/node-test-commit/75761/

00:00:00.379 Honoring refspec on initial clone
00:00:00.423 Cloning repository git@github.com:nodejs/node.git
00:00:00.423  > git init /home/iojs/build/workspace/node-test-commit # timeout=10
00:00:00.427 ERROR: Error cloning remote repo 'origin'
00:00:00.427 hudson.plugins.git.GitException: Could not init /home/iojs/build/workspace/node-test-commit

[targos]

/cc @nodejs/build-infra

I'm in a meeting the whole day and cannot investigate this.

[mhdawson]

@ryanaslett is this something you could take a look at?

[ryanaslett]

I did some preliminary investigation into this and it appears as though all three of our jenkins workspace machines can no longer connect to github.

All the clone urls are git@github.com, and result in

git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So either the url's changed in the build and they used to be https urls, or something changed on the workspace machines to remove the private keys.

[mhdawson]

Earlier issue where we had trouble with connectivity from jenkins to GitHub, but this looks different- #3254

[mhdawson]

Being looking at the keys/setup with Ryan to try to understand what might have changed, have followed some threads but no answer yet.

[mhdawson]

Found this issue which looks like it might be the same issue after looking at the error more closely and some googling - #3614

Looks like packages were updated yesterday (by looking at /var/log/dpkg.log) on test-ibm-ubuntu2204-x64-3 were some of the failures were being seen.

Possible that it ran/failed so fast that all jobs were running failing there.

Have restarted the jenkins agent and job seems to be running now. Looks ike it was just that that git just failing to run from java running the agent after the system updates like was the case reported earlier in #3614.

Restarted the agent on the 3 workspace machines that I see being associated with the node-test-commit job.

[mhdawson]

This job seems to be running ok - https://ci.nodejs.org/job/node-test-commit/75781/

Will leave this open until we confirm that we see jobs running ok across the different workspace machines.

[ljharb]

still seeing the same issue on https://ci.nodejs.org/job/node-test-pull-request/63546/

[aduh95]

The issue for node-test-commit is resolved, but there are still other jobs impacted (node-test-commit-linux and node-test-commit-custom-suites-freestyle (test-worker))

[targos]

I updated and rebooted all 4 ubuntu2204 hosts

[aduh95]

That seems to have fixed it, https://ci.nodejs.org/job/node-test-pull-request/63552/ doesn’t have the issue. Closing now, thanks everyone.

[ljharb]

fwiw, https://ci.nodejs.org/job/node-test-pull-request/63557/ still has the same issue.