Debian versions and Python 3 (Node.js 16 and later)

[richardlau]

Currently in the CI we have Debian 8 (Jessie) and 9 (Stretch) on x64 and arm (through Docker templates armv7_jessie.Dockerfile.j2 and armv7_stretch.Dockerfile.j2). Debian 8 is only used for testing Node.js 10 (End-of-Life next month) and 12:

build/jenkins/scripts/VersionSelectorScript.groovy

Line 31 in c2732c3

[ /debian8/, anyType, gte(13) ],

We're trying to get Python 3 installed on the CI across the platforms that we build/test Node.js 16 onwards, #2507, to unblock nodejs/node#37764. There's also nodejs/node#36691 which removes support for building with Python 2 and Python 3.5. This will be an issue for Stretch as the latest version of Python 3 from the default repositories in 3.5.3: https://packages.debian.org/stretch/python3

The Pi's are actually running Debian 10 (Buster) and have Python 3.7.3 available/installed (https://packages.debian.org/buster/python3). However tests are executed in the aforementioned Docker containers (Stretch for Node.js 12 and later).

I think our options are (feel free to suggest anything I might have overlooked):

1. Add Debian 10 (Buster) into the CI. Including Docker files for the Pi's. Switch testing for Node.js 16 to these.
2. Compile a later version of Python 3 (in ansible either directly or via pyenv) for Debian Stretch.

Debian 8 actually went out of LTS in June 2020 (https://www.debian.org/releases/jessie/) so maybe we could remove Debian 8 from our CI to free up resources for Debian 10? Going to Debian 10 would also give us gcc/g++ 8.3.0 (https://packages.debian.org/buster/gcc) which is what we've raised the minimum supported gcc/g++ level to (Debian 9 is on gcc/g++ 6.3.0, https://packages.debian.org/stretch/gcc) so I'd lean towards adding Debian 10.

Thoughts?

cc @nodejs/build-infra

[rvagg]

Re Pi: Yes, we need to upgrade the platforms we test on and remove old ones, I'd even be happy removing all we have and replacing the entire fleet with containers running the latest debian (only). This requires quite a bit of work unfortunately, maybe it's improved in recent times (probably), but even sourcing solid base images has been a problem when trying to build these - even rebuild them, so I'm usually hesitant when I have to build images from scratch using our Dockerfile templates!
Re x64 debian - be ruthless. Let's retire some old machines and replace them with newer Debian. We'll still have good coverage for older release lines and we rarely (if ever?) catch problems for old release lines on a specific Linux OS and not on others--and having Debian + Ubuntu does double up a little since they're so similar.

[richardlau]

Re. x64 debian:
We currently have four debian 8 CI hosts:

We no longer build/test on 32-bit Linux, so my plan is to remove lingering references debian8-x86 from our CI jobs and then replace test-softlayer-debian8-x86-1 with Debian 10 x64 -- I'm assuming I won't be able to upgrade Debian 8 x86->Debian 10 x64 and will have to remove the existing host and create a new one in IBM Cloud. I'm also proposing to replace one of the rackspace hosts so we (temporarily) end up with two Debian 8 x64 hosts and two Debian 10 x64 hosts. We can then later replace the remaining Debian 8 hosts when we are satisfied that the Debian 10 ones are working correctly.

[rvagg]

👍 you may be able to re-image existing machines, saves the firewall updates on CI hosts at least

[richardlau]

test-softlayer-debian8-x86-1 has been re-imaged (after I removed all remaining references to debian8-x86 in the job configs) and is now test-softlayer-debian10-x64-1. Associated inventory/ansible updates: #2612

I'll look at replacing one of the rackspace debian8-x64 hosts tomorrow. Once we have two working debian 10 hosts I'll add them to the node-test-commit-linux and update the VersionSelectorScript to stop testing master/Node.js 16 on Debian 8/9.

[richardlau]

Status here is that we have two working Debian 10 hosts. I'm planning to wait until after Node.js 10 goes End-of-Life before removing the remaining two Debian 8 hosts.

[nschonni]

Since 10 is now EOL, did those 8 hosts get removed?
Just came across this one, since I was seeing if there was an issue around adding the new Debian 11 https://www.debian.org/releases/bullseye/

[richardlau]

They have not -- replacing/upgrading them is still a TODO.

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.