8352044: Add --with-import-jvms to configure

[magicus]

We should allow pre-built JVMs to be included in a build, so they are just copied into place, and the jvm.cfg file properly updated.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8352044: Add --with-import-jvms to configure (Bug - P3)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24063/head:pull/24063
$ git checkout pull/24063

Update a local copy of the PR:
$ git checkout pull/24063
$ git pull https://git.openjdk.org/jdk.git pull/24063/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 24063

View PR using the GUI difftool:
$ git pr show -t 24063

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24063.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back ihse! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@magicus The following labels will be automatically applied to this pull request:

• build
• core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (e3090e8f)

[shipilev]

This change copies libjvm.so and sibling .jsa files, right?

If so, then one thing is missing: regenerating CDS archives that have opinions on modules filesizes/dates for fingerprinting their CDS archives. My frankensteining scripts do that by invoking new JVM explicitly with -Xshare:dump. But build system should already know how to do that, as it does it at the end of the build.

Try to import a JVM and do java -Xshare:on Hello?

[magicus]

This change copies libjvm.so and sibling .jsa files, right?

No, it doesn't -- only libjvm.so. .jsa files complicate the situation. Hm.

Maybe we should point to an import JVM directory, and copy it with all its files, and not just try to single out the libjvm.so. That will allow imported JVMs to tag along any kind of files they want.

[magicus]

Try to import a JVM and do java -Xshare:on Hello?

The problem is that we can't really count on -Xshare being supported on an imported JVM.

[shipilev]

No, it doesn't -- only libjvm.so. .jsa files complicate the situation. Hm.

Aha. Importing CDS archives without regenerating them is futile, IIRC: they would never load properly, CDS would get disabled, and we would just carry dead weight.

Maybe we should point to an import JVM directory, and copy it with all its files, and not just try to single out the libjvm.so. That will allow imported JVMs to tag along any kind of files they want.

Yes. There are also debuginfo files that you certainly want.

The problem is that we can't really count on -Xshare being supported on an imported JVM.

This is suggestion for testing CDS support. We don't need to do -Xshare:on tests from a build system.

[dholmes-ora]

@magicus I'm trying to see the big picture here of all the changes you are proposing and I'm really not seeing it. This seems to be making it more difficult to create a JDK/JRE with multiple VMs.

[AlanBateman]

The discussion on jdk-dev was useful but I don't think adding --with-import-jvms is the right direction. It's too fragile and loose to import from a build created somewhere else.

I can relate to Aleksey's case where it might be useful to have release + debug builds in the same run-time image. Having java launcher support selecting the fastdebug libjvm, and using jlink to include/exclude the debug VM, can become part of a good story.

I'm less sure that builds that want to produce server + minimal or server + zero or other combinations like this are important or compelling in 2025. In the case of minimal then it may be more useful for the build to just produce the packaged modules (JMOD) rather than an image. When targeting small/embedded environments then this is what you need as a JDK build with all modules + minimal VM doesn't really make too much sense.

[shipilev]

For completeness/reference, this is my JDK frankensteining script that is used for producing builds.shipilev.net bundles: bundle-multi-jdk.sh.txt. That would be one of the workflows that new build support feature would need to help.

[magicus]

The discussion on jdk-dev was useful but I don't think adding --with-import-jvms is the right direction. It's too fragile and loose to import from a build created somewhere else.

Why's that? It's no more loose than just "importing" a jtreg jar from "somewhere else". Nor is it any more fragile than any other part of the build system. In fact, I think you seriously underestimate how fragile the current solution is, where we have to manage multiple hotspot builds. I've lost count on how many times we've had to solve bugs related to this. That is a very weird quirk in the build system, that has ramifications all over, and making all changes related to hotspot being much harder and riskier.

Also, to quote what I just wrote in a JBS issue:

This is all not really about removing any functionality. It is just about shifting the cost of doing this odd combinations to the distributors who still want to support them, instead of letting the entire JDK build ecosystem pay the price.

Any distributor who wants to build a JDK with both minimal and server will still be able to do that. But since that is a niche case, it stands to reason that they must add just a tiny bit of complexity to their build scripts to achieve this.

But as a result of removing this complexity from the build system in the JDK, it will allow us to unlock a lot of well-needed functionality, such as decoupling the gtest build from the hotspot build. This in turn will lead to faster builds, and the ability to use gtest for testing of native libs (outside Hotspot) in the JDK. I can't see how it is by any mean worth paying the price of missing out on this functionality, just to keep distributors from having to modify their build scripts, for a combination that I think everyone agrees are at least on its way out.

[magicus]

And to be absolutely clear: this PR is just about adding new functionality that was not present before.

A follow-up PR for JDK-8352045 will actually remove functionality from the build system. But, and once again, let me be absolutely clear, the intention is that even after that PR, any distributor who wants to build an image with multiple JVMs will still be able to do so. The generated bits will be identical before and after JDK-8352045; the only difference is that you are going to need a "pre-build" step for each additional JVM you want to add, where the pre-build step basically consists of calling bash configure --with-jvm-variant=<my variant> && make hotspot. If you add this simple step, you are going to get the exact same bits in the image.

[magicus]

In theory, I could even program a make target to call configure --with-jvm-variant=<my variant> --with-conf-name=support-for-<my variant> && make hotspot CONF=support-for-<my variant> but that's kind of silly, and looks like babysitting and insulting the intelligence of distributors...

[erikj79]

Do you really need to call MakeDir outside of a recipe? Won't SetupCopyFiles recipes create this directory?

[magicus]

I assumed it would, but it failed, and I looked at the code and found no clear evidence that it does, so I added this. It might have been some other issue, so I can double-check again, if you are certain that SetupCopyFiles should create needed directories.

[erikj79]

SetupCopyFiles uses install-file by default and that macro calls MakeTargetDir.

[erikj79]

If we want to support CDS archives for the imported JVMs, then we would need to modify the loop in Images.gmk so that it covers them as well. I think that makes sense to do.

[magicus]

If we want to support CDS archives for the imported JVMs, then we would need to modify the loop in Images.gmk so that it covers them as well. I think that makes sense to do.

Yes, I agree. That, and importing the whole directory, is the two changes I have on my mental todo list for this PR.

[AlanBateman]

And to be absolutely clear: this PR is just about adding new functionality that was not present before.

Sure but I think it's also an attractive nuisance. The VM is very tightly coupled to java.base and a few other core modules. I don't think the build should be supporting grabbing libjvm from another build or another location. Point taken that tools to build and test the JDK need to come from somewhere but it's not as tightly coupled as we have with the proposal here.

For the folks targeting embedded/small environments then I think the right thing is to publish the packaged modules (JMOD files) for the target platform and then use jlink to create the run-image with the right VM + small set of modules that are appropriate for the target environment.

[erikj79]

For the folks targeting embedded/small environments then I think the right thing is to publish the packaged modules (JMOD files) for the target platform and then use jlink to create the run-image with the right VM + small set of modules that are appropriate for the target environment.

Just to make sure I'm understanding you correctly. Are you proposing that instead of building a single JDK distribution with multiple JVMs, they would build a separate JDK for each alternative JVM configuration and publish java.base.jmod from each of them for end users to pick from when generating their run-images?

[AlanBateman]

Just to make sure I'm understanding you correctly. Are you proposing that instead of building a single JDK distribution with multiple JVMs, they would build a separate JDK for each alternative JVM configuration and publish java.base.jmod from each of them for end users to pick from when generating their run-images?

I think there is a compelling case for having a release + fastdebug build of a server libjvm in the same run-time image. I'm dubious about server + minimal in the same run-time image, it's just too weird unless users of minimal VM are also running with --limit-modules to use a reduced set of modules before running jlink to create the target run-time image for the small environment.

For embedded environments then it could be compelling to have the the packaged modules (JMOD files, and not just java.base) published for several OS/architectures. There's a connection to cross building here. Once you have the packaged modules then someone could run jlink (on a main-stream platform) to produce the run-time containing the small set of modules that are appropriate for the target environment.

[bridgekeeper]

@magicus This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[magicus]

KEEPALIVE

[bridgekeeper]

@magicus This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[magicus]

/keepalive

[openjdk]

@magicus The pull request is being re-evaluated and the inactivity timeout has been reset.

[bridgekeeper]

@magicus This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!