UnixPb: Install x64 alpine bootjdks where available

[Haroon-Khel]

• commit message has one of the standard prefixes
• faq.md updated if appropriate
• other documentation is changed or added (if applicable)
• playbook changes run through VPC or QPC (if you have access)
• for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

ref #2564

As far as I can see, the alpine jdk build jobs run the build on our alpine docker images, so I have updated what I think are the appropriate files. The alpine pr check will be the judge :)

I've kept some of the oddities, such as JDK7_BOOT_DIR="/usr/lib/jvm/zulu8 despite being a jdk7 bootjdk I've kept it as a jdk8 binary.

[karianna]

I think we're starting to support Alpine on Aarch64? Does this invalidate that?

[sxa]

I think we're starting to support Alpine on Aarch64? Does this invalidate that?

It would depend if we're happy with a nightly going in as we haven't released a GA version on that platform yet for any version.

[sxa]

@Haroon-Khel I think for now I still want to install the Zulu JDK8/11 (as a minimum) on aarch64 but I think this PR might be removing that.

[sxa]

@Haroon-Khel FYI #2568

[sxa]

@Haroon-Khel will this not override JDK7_BOOT_DIR etc. on Alpine/aarch64 which will still have zulu boot JDK and won't have anything /usr/lib/jvm/jdk8 etc. for now?

[Haroon-Khel]

So now it should install temurin jdks 8 11 and 17 for both alpine x64 and aarch64 and leave the rest as zulu
To install the aarch64 binaries, I have placed 'static' jenkins nightly links which I do believe the alpine images are able to download. However I do not know if these links are the best option as I do not know if they expire. Open to review

[Haroon-Khel]

The code for installing the aarch64 is a bit messy as it uses a nightly link and not a release link. This is temporary until we have a ga aarch64 alpine linux binary

[Haroon-Khel]

Does the alpine3 docker image get pushed to dockerhub after the pr gets merged or after the github workflow passes? @sxa @karianna

[sxa]

I hope it's after the merge takes place 👀

My only concern just now is that I'm not convinced that the Temurin 11 alpine/aarch64 JDK is stable at the moment (If you look at https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-alpine-linux-aarch64-temurin/ you can see that things are getting 'stuck' until they hit the 10 hours timeout) so I'm tempted to suggest leaving that one as zulu for now (or potentially installing both for 11)

[sxa]

However I do not know if these links are the best option as I do not know if they expire.

Yeah those links from jenkins will expire after a couple of builds as it takes up too much space on the jenkins server to retain them - even though it says lastSuccessfulBuild the artifacts on those jobs are datestamped so the old names will disappear within a few days. You could pull them directly from the github repo e.g. https://github.com/adoptium/temurin17-binaries/releases/download/jdk17u-2022-05-27-19-32-beta/OpenJDK17U-jdk_aarch64_alpine-linux_hotspot_2022-05-27-17-01.tar.gz which are always retained.

[Haroon-Khel]

I've changed the links for the aarch64 alpine jdk 8 11 and 17 binaries to links from our github/temurin*-binaries repos, so these shouldn't expire

My only concern just now is that I'm not convinced that the Temurin 11 alpine/aarch64 JDK is stable at the moment (If you look at https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-alpine-linux-aarch64-temurin/ you can see that things are getting 'stuck' until they hit the 10 hours timeout) so I'm tempted to suggest leaving that one as zulu for now (or potentially installing both for 11)

The binary links that I've added for the aarch64 binaries do not get overridden do they? ie, the links in https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u-2022-05-28-05-51-beta do not get overridden by newer builds? If so then I do not see how the failing build job will impact the installation of the binary. Unless you mean that the failing build job causes you to lose confidence in the binary itself?

[Haroon-Khel]

I hope it's after the merge takes place 👀

This may be the case. I've pulled the latest adoptopenjdk/alpine3_build_image and the changes are not in place

[sxa]

The binary links that I've added for the aarch64 binaries do not get overridden do they? ie, the links in https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u-2022-05-28-05-51-beta do not get overridden by newer builds?

Correct

If so then I do not see how the failing build job will impact the installation of the binary. Unless you mean that the failing build job causes you to lose confidence in the binary itself?

At the moment the Temurin 11 builds on Alpine/aarch64 build and publish but do not work correctly, so I don't want to rely on those builds as the boot JDK for building the nightlies of JDK11. At present we're using a non-Temurin build for the boot JDK, and if we install the Temurin one we're risking making it more unstable as it's known to have problems.

(It probably needs to be tested to be honest - check whether running a build with zulu or Temurin as the boot JDK makes any difference to the build)

[Haroon-Khel]

(It probably needs to be tested to be honest - check whether running a build with zulu or Temurin as the boot JDK makes any difference to the build)

I'll give this a try

[sxa]

(It probably needs to be tested to be honest - check whether running a build with zulu or Temurin as the boot JDK makes any difference to the build)

I'll give this a try

We have quite a few options now for bootstrapping JDK11 - There's the Alpine-supplied openjdk 11 package, a zulu one, or a Temurin nightly :-)

[Haroon-Khel]

@sxa After two attempts (first attempt hung), I got jdk11 to build on dockerhost-equinix-ubuntu2004-armv8-1, currently the only machine that builds jdk11 aarch64 alpine linux.

I also got it to build on https://ci.adoptopenjdk.net/computer/docker-equinix-ubuntu2004-armv8-1/, and have now added the alpine-linux label to it so that it may get picked up by some of the alpine jdk build jobs

To summarise, I think the instability of the jdk11 aarch64 alpine linux build job is due to infra related issues on dockerhost-equinix-ubuntu2004-armv8-1. The timed out test jobs in adoptium/temurin-build#2961 also ran on that machine. Either way, I think using a temurin nightly as a bootjdk for the alpine images is fine

[sxa]

Interesting ... Do you have Grinder runs from it running on the other machine?

[Haroon-Khel]

Good point.
I've set up https://ci.adoptopenjdk.net/computer/test-docker-alpine311-aarch64-1/ and https://ci.adoptopenjdk.net/computer/test-docker-alpine312-aarch64-1/ on https://ci.adoptopenjdk.net/computer/docker-equinix-ubuntu2004-armv8-1/. It seems the only aarch64 test alpine containers were hosted on dockerhost-equinix-ubuntu2004-armv8-1.

I've kicked off some grinders on the alp311 and 312 containers

Sanity openjdk https://ci.adoptopenjdk.net/job/Grinder/4970/console
Extended functional https://ci.adoptopenjdk.net/job/Grinder/4971/console
Sanity functional https://ci.adoptopenjdk.net/job/Grinder/4972/console
Sanity system https://ci.adoptopenjdk.net/job/Grinder/4973/console
Sanity perf https://ci.adoptopenjdk.net/job/Grinder/4974/console
Extended system https://ci.adoptopenjdk.net/job/Grinder/4975/console

Hoping they dont time out

[Haroon-Khel]

The tests look unstable even on the second machine. So I've returned jdk11 back to being installed from the alpine repos, but I have kept its link as /usr/lib/jvm/jdk-11 instead of zulu11 so that the links can be consistent on x64 and aarch64

[Haroon-Khel]

ping @sxa for review

[Haroon-Khel]

I'm not quite sure what the linter is complaining about

Error: schema [{'os_img': 'aix/ppc64/jdk'}, {'heap': 'normal'}, {'impl': 'hotspot'}, {'new_baseurl': 'https://api.adoptium.net/v3/binary/latest'}, {'new_vendor': 'eclipse'}, {'baseurl': 'https://api.adoptopenjdk.net/v3/binary/latest'}, {'vendor': 'adoptopenjdk'}, {'project': '{{ heap }}/{{ vendor }}?project=jdk'}] is not of type 'object'
Error: schema None is not of type 'boolean'
Warning: ignore-errors Use failed_when and specify error conditions instead of using ignore_errors.
Error: schema 1 is not of type 'string'
Error: schema [{'ansible_python_interpreter': '/usr/local/python2/bin/python2.7'}] is not of type 'object'
Warning: var-spacing Jinja2 variables and filters should have spaces before and after.
You can skip specific rules or tags by adding them to your configuration file:
# .config/ansible-lint.yml
warn_list:  # or 'skip_list' to silence them completely
  - experimental  # all rules tagged as experimental
  - var-spacing  # Jinja2 variables and filters should have spaces before and after.
Finished with 1 failure(s), 5 warning(s) on 414 files.
Error: Process completed with exit code 2.

[sxa]

I'm not quite sure what the linter is complaining about

Seems to be happening on other PRs too, so unlikely to be related to anything here.

[sxa]

I think I'm ok with this now. Will aim to merge tomorrow (Don't want to do it before the nightly builds...