Provide download test capability

[sxa]

I thought we already had an issue covering the production of an IVT (Installation Verification Test) but apparently not. We should have a mechanism for verifying the artifacts we publish up including verifying checksums and ensuring that what we ship is as secure as possible.

I have put together a job in the private jenkins instance (for now) which performs the following checks. If anyone has ideas for other things that we could do please feel free to add them but I believe this is a good start:

• tag supplied as parameter to the job
• Job downloads artefacts from the appropriate temurinXX-binaries repository
• GPG signatures and SHA256 checksums verified for all files
• Attempt to count the number of files in each tar file (Needs additional check to ensure there is no error code, or a zero number!)
• The job is defined to run on a Linux/aarch64 machine - separately extract the JDK and JRE tarball and run java -version against them then check for the versions of GCC and GLIBC (Needs to be adjusted for other versions as it explicitly checks 2.17/11.2 just now so will fail in JDK<20)
• Runs the cyclonedx validation tool on the SBOM
• SBOM validation as per Provide validation of the SBOM content #3484

Potential extra todos:

• Extract and check all Linux ones to check GLIBC/GCC version and/or compare to the information in the SBOM
• Make the script nicer to read
• Check that the location where the binaries are stored by the EclipseMirror job has identical content (Although we don't publicise that location...)
• Figure out where in the process this should be invoked from :-)

The current plan is to use this to verify the October releases (the first one that will have suitable content in the SBOMs) and then do any fixes required before putting this job under version control (the test cycle for changes was quicker having it inlined for now)

[sxa]

Observation: The Linux debugimage files have significantly fewer fiels on Linux tha
n on other platforms:

=== OpenJDK11U-debugimage_aarch64_linux_hotspot_11.0.18_10.tar.gz 78
=== OpenJDK17U-debugimage_aarch64_linux_hotspot_17.0.9_9.tar.gz 71
=== OpenJDK21U-debugimage_aarch64_alpine-linux_hotspot_21.0.1_12.tar.gz 71
=== OpenJDK8U-debugimage_aarch64_linux_hotspot_8u392b08.tar.gz 100

=== OpenJDK17U-debugimage_aarch64_linux_hotspot_17.0.9_9.tar.gz 71
=== OpenJDK17U-debugimage_aarch64_mac_hotspot_17.0.9_9.tar.gz 421
=== OpenJDK17U-debugimage_arm_linux_hotspot_17.0.9_9.tar.gz 73
=== OpenJDK17U-debugimage_ppc64le_linux_hotspot_17.0.9_9.tar.gz 71
=== OpenJDK17U-debugimage_s390x_linux_hotspot_17.0.9_9.tar.gz 69
=== OpenJDK17U-debugimage_x64_alpine-linux_hotspot_17.0.9_9.tar.gz 70
=== OpenJDK17U-debugimage_x64_linux_hotspot_17.0.9_9.tar.gz 72
=== OpenJDK17U-debugimage_x64_mac_hotspot_17.0.9_9.tar.gz 421

For mac its over 400 (Also there's a variation in the JDK17 counts on different linuxes which is interesting). I would have expected them to be similar (one per executable or shared library, which naively should be similar on all platforms). Maybe linux is excluding things in jmods or something ... Should investigate further

[sxa]

Job is now complete and should be ready for "prime time" on the main CI server as it successfully validates all of the October releases that have been published to date
I've compared this with the changes already proposed in the docs at https://github.com/adoptium/temurin-build/pull/3455/files and done updates to that PR accordingly

Noting that

• AIX does not mention the compiler version, only that it isIBM XL C/C++
• Windows does not list the build version which is now important from a reproducibility perspective.
• These checks are a "point in time" and are not intended to verify older versions that may have used earlier compilers.
• The freetype explicit version checks have been removed as they're a bit inconsistent just now
• These checks should therefore probably be versioned/branched with the builds.
• It's "functional" rather than "nice" as scripts go, so could do with a good amount of tidying up in the future.

[sxa]

Ref the tidying up here are some ideas for the future:

1. Need a better method for verifying the temurin-build SHA - the current method doesn't work if the SHA hasn't been tagged.
2. Allow it to run on something other than Linux/aarch64 (cyclonedx, jq, and the java -version check only work on there for now)
3. Alternatively for 1 some of the GLIBC/GCC checks with strings could be done regardless of the platform
4. gpg signature checks could do with being a little less verbose (we only really need to know when they fail)
5. Similarly, the whole of validateSBOMcontent.sh could do with being less verbose by default.
6. Better tolerance if only tar.gz or zip files are in the release.
7. Maybe separate into functions
8. It could be converted to a staged pipeline I guess to make it a bit nicer to look at and update the job name.

[sxa]

Job is live at https://ci.adoptium.net/job/build-scripts/job/release/job/download_and_sbom_validation including runs for all four of the October releases but will need switched to run from the main temurin-build master branch by default instead of my fork once the PR referenced above is merged.