ARM cluster move

[rvagg]

Sad news folks, I have to physically move the ARM cluster today and the internet connection where it's moving to isn't properly installed yet! I have a temporary connection ready but I'm paying by the GB for it so I can't hook up normal test runs.

There are some ARMv7 and ARMv8 machines in Jenkins (and ARMv7 in release) that aren't physically in the same place (i.e. they are hosted at Scaleway and miniNodes) so they won't be impacted.

So here's what I'm going to do:

• ARM test and releases via Jenkins will be unavailable from around 4am UTC / 8pm Pacific for at least a few hours, possibly longer (up to 24 hours) as I hack together the temporary connection.
• I'll unhook the relevant machines from Jenkins so nobody should be obviously impacted, you just won't get the complete runs you normally do.
• When reconnected, I'll only be connecting release machines until I have a proper connection or decide that it'll be too long to wait and come up with an alternative.

I'll keep this thread updated as I make progress.

@nodejs/collaborators

[mikeal]

Will we get a new picture of the cluster once it is in its new home?

[rvagg]

release machines are back online for now, armv6 and armv8, others are off, no ETA on proper connection yet

[Trott]

So with no ETA for the return of the Raspberry Pi cluster:

For the jobs that are stalled waiting for the Raspberry Pi farm, will they kick off tomorrow or whenever the farm comes back online? Or probably not and the jobs should just be canceled now?

Land stuff without Raspberry Pi test results in CI? Or wait for the Raspberry Pi cluster to come back?

/cc @nodejs/ctc

[joaocgreis]

armv7-ubuntu1404 and armv8-ubuntu1404 were removed by @rvagg from the node-test-commit-arm job but node-test-commit-arm-fanned was left in place, possibly forgotten. I think it's better to cancel. I'll look for a way to remove the whole job.

EDIT: Just disabling the job woked, it is properly skipped by node-test-commit. Also disabled git-rpi-clean.

[rvagg]

Sorry, I thought I removed node-test-commit-arm-fanned. There shouldn't be any queued jobs, if there are then I've messed up!

[rvagg]

Thursday the 9th is the date I've been given for finalising this internet connection. Apparently there are some technical challenges (also I think some administrative incompetence but that's to be expected when dealing with large telcos!).

[rvagg]

Bad news .. I've been notified there are network problems in the area (monopoly government-provided internet infrastructure, yay) and it's been deferfed for another week. If it goes through then it should be up on the 16th of this month.

[thefourtheye]

We have three releases and we might get RCs out soon. Should we hold them till this setup is back up? We cannot release binaries without testing them, right?

[italoacasas]

I have two questions:

• something I(we) can do to help right now?
• something we can prepare(plan) in the case that this happens again in the future, like for example a storm, etc.

[mhdawson]

The LTS releases are planned for Feb 21st so availability on the 16 may not affect those directly. It may affect plan RC's, in that case the question would be if the changes going in that we wanted validation through the RC would be ARM only or can be adequately covered by use on other platforms.

In terms of testing for the Current release, I wonder if the binaries could be tested manually by somebody with access to the release machine logging in and running the tests. That might take a while to run thought since it would be on the single machine instead of fanned like it is in the regular jobs.