Wrong folder name in published MacOS x64 builds of static libraries for 23-EA

[zakkak]

What are you trying to do?

Build Mandrel using Temurin 23 EA builds with static libs downloaded from https://github.com/adoptium/temurin23-binaries/releases/download/jdk-23%2B4-ea-beta/OpenJDK-static-libs_x64_mac_hotspot_ea_23-0-4.tar.gz

Expected behaviour:

Static libraries should be placed under lib/static/darwin-amd64 and the build should pass.

Observed behaviour:

Static libraries are placed under lib/static/darwin-arm64 instead and the build fails.

Any other comments:

Despite the folder naming the static libraries appear to target amd64.
Renaming the directory in the local installation resolves the issue.

[jerboaa]

/cc @Haroon-Khel Could this be caused by doing cross builds on arm for mac os x x86_64 these days?

[Haroon-Khel]

That does seem likely to be the cause.

Looks like the path name is decided here

temurin-build/sbin/build.sh

Line 1329 in a352e08

staticLibsDir="Contents/Home/lib/static/darwin-${osArch}"

[adamfarley]

Taking a look. :)

[adamfarley]

TLDR

We're confusing the build machine arch with the target arch. Will create a PR to fix this for the static libs and SBOM.

Other issues we've found will be discussed further down, and moved into a separate PR.

Investigation details

osArch is set by the BUILD_CONFIG parameter OS_ARCHITECTURE, which is set to "arm64" (even on x64 builds), like Haroon suggested.

OS_ARCHITECTURE is set by the temurin-build config_init.sh script, which sets it to the "arch" parameter.

config_init.sh inherits parameters from makejdk_any_platform.sh

The temurin-build readme specifies arch as:

"This tag identifies the architecture the JDK has been built on and it intended to run on."

This conflates the build platform and the runtime platform, causing our headache.

(BUILD_FULL_NAME seems to do the same thing, and we do that to ourselves from OS_ARCHITECTURE in configureBuild.sh, but it doesn't look like we use BUILD_FULL_NAME for anything. Vestigial?)

So, yeah, hmm. Solving this problem in a cheap, hacky way seems simple, but I want to be sure there's not an underlying problem here.

Ok, JDK8 appears to set OS_ARCHITECTURE to x86_64 with no ill effects.

Further digging indicated that we're not doing some of the cross-compilation prep on JDK11+, as per this line:

    export MAC_ROSETTA_PREFIX="arch -x86_64"

The question is: Why isn't this line breaking the build on JDK8?

  local arch=$(uname -m)
...
  BUILD_CONFIG[OS_ARCHITECTURE]=${arch}

Turns out uname -m returns x86_64 on JDK8 builds, but returns arm64 on JDK11+ builds. Both are run on orka arm nodes so....huh.

Turns out Orka uses Rosetta for JDK8 builds to fake running on an x86_64 architecture, which is why uname -m returns x86-64 for JDK8 but arm64 for everything else.

We can't use Rosetta for JDK11+ (xcode version problems), so we'll fix this issue by correcting build.sh to use the correct architecture variable.

Good thing we did the research first though, as we'd not have noticed the SBOM mix-up otherwise.

[sxa]

@andrew-m-leonard this is the issue we spoke about earlier today. It's not limited to JDK23 so will be present in the other releases we've shipped form the cross-compiled mac builds.
Cc @Haroon-Khel

[adamfarley]

@Haroon-Khel

There seem to be several issues coming out of this, which are likely an issue each. Please review and let me know your thoughts:

• Any thoughts on why we're generating static libs if we don't value it enough to test it?
• We're conflating (mistakenly combining) the target OS and the build/host OS, in several places, mainly:
  • In the SBOM (OS architecture) we use the build OS when we should use the target OS.
  • In config_init, we set the "OS_ARCHITECTURE" to the "uname -m" output, and then we use it as the target architecture in at least a few places. Which prevents a clean crosscompilation.
• The code has BUILD_CONFIG[OS_ARCHITECTURE], ARCHITECTURE, MACHINEARCHITECTURE, arch, and more! We should just have two global values (target and build), to prevent this headache in the future.

[adamfarley]

Ok, here's a short-term fix for the Static Libs folder issue, as well as the SBOM issue.

#3619

In the long run we should separate the target and build architectures as BUILD_CONFIG elements.

[jerboaa]

• Any thoughts on why we're generating static libs if we don't value it enough to test it?

Static libs are mostly used by GraalVM builds. GraalVM is based on LabsJDK (a OpenJDK fork). Mandrel is based on OpenJDK (vanilla). So we raised this issue since we build mandrel with it in the mandrel-packaging PR checker. So we notice eventually. As we did and filed the issue.

Edit: We are currently working around this with graalvm/mandrel-packaging@e88c8b3

[jerboaa]

This seems to be a problem for JDK 21, JDK 22 and JDK 23 builds.

[adamfarley]

This issue has now been resolved. Correctly-named ea versions of static libs can be found here:

• jdk21 - Next release version in April.
• jdk22 - Next release version in March.
• jdk23 - Next release version in September.

The fix resolving this issue in the short term can be found here: #3619
The long-term fix will be worked on here: #3648

Closing this issue.

[jerboaa]

Thanks. @adamfarley Are next ea builds for 23, 22 and 21 expected to have the fix?

[adamfarley]

All builds produced by Adoptium frameworks from this point onwards should contain this fix.

[jerboaa]

All builds produced by Adoptium frameworks from this point onwards should contain this fix.

Thanks for the info.

[sxa]

Perhaps we should add this to the release blog?

[jerboaa]

Perhaps we should add this to the release blog?

In what way? As a known issue?

[sxa]

Perhaps we should add this to the release blog?

In what way? As a known issue?

Yep and one that we're acknowledging and it will be fixed in the next release (or the ea/nightlies for anyone interested)