8346719: Add relaunchers to the static JDK image for missing executables

[magicus]

In the static JDK image, a single humongous java executable is generated, and no other launcher, such as javac. This makes it impossible to run our jtreg tests, which assume these are present.

The solution is fortunately simply: we just need to add a bunch of trivial launchers, which are thin wrappers that execute the main java binary, with the proper arguments. This will result in the same behavior as the normal dynamic launchers, only that we will need to take the detour of launching another process instead of calling directly into the JLI library.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Error

 ⚠️ 8346719 is used in problem lists: [test/langtools/ProblemList-StaticJdk.txt, test/lib-test/ProblemList-StaticJdk.txt, test/jdk/ProblemList-StaticJdk.txt]

Warnings

 ⚠️ Patch contains a binary file (src/java.desktop/share/classes/java/awt/doc-files/BorderLayout-1.gif)
 ⚠️ Patch contains a binary file (src/java.desktop/share/classes/java/awt/doc-files/FlowLayout-1.gif)
 ⚠️ Patch contains a binary file (src/java.desktop/share/classes/java/awt/doc-files/GridBagLayout-1.gif)
 ⚠️ Patch contains a binary file (src/java.desktop/share/classes/java/awt/doc-files/GridBagLayout-2.gif)
 ⚠️ Patch contains a binary file (src/java.desktop/share/classes/java/awt/doc-files/GridLayout-1.gif)
 ⚠️ Patch contains a binary file (src/java.desktop/share/classes/java/awt/doc-files/GridLayout-2.gif)

Issue

• JDK-8346719: Add relaunchers to the static JDK image for missing executables (Bug - P3)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24380/head:pull/24380
$ git checkout pull/24380

Update a local copy of the PR:
$ git checkout pull/24380
$ git pull https://git.openjdk.org/jdk.git pull/24380/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 24380

View PR using the GUI difftool:
$ git pr show -t 24380

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24380.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back ihse! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@magicus The following labels will be automatically applied to this pull request:

• build
• core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[magicus]

A bit of discussion/rationale for this PR:

• I did some preparation of the main launcher in 1d6c2ea. The old launcher code had a lot of weird stuff in define.h (which was not just defines, as the name implies). I simplified this and inlined it directly into main.c. I also renamed variables prefixed const_, because most of them will not be that after this PR, and it did not seem to bring much value to point this out in the name.
• Any changes in main.c apart from these are within #ifdef STATIC_BUILD only, and should not affect the normal launcher.
• The main launcher need some additional information to be able to launch any tool except the main "java" itself. Normally this is "hard-coded" into different builds of main.c by the use of compiler defines passed by the build. The approach I used for the static launcher is to pass this information as special system properties, with a name starting with javaLauncher. This will be treated as normal system properties by the launcher, but the additional code activated in the static build of the launcher will also interpret these values, and set the variables corresponding to how they would have been hardcoded in the dynamic launcher.
• This could have been achieved by using shell script/bat file relaunchers instead. However, that seemed more risky, since it would change the launchers from native binaries to text files. On Windows, for instance, other programs might look for "javac.exe" and not consider "javac.bat" to be acceptable.
• Finally, a commonly asked question when I developed this has been: "Could you not just symlink the java binary with different names, and then dispatch it depending on how it was called?". This is a common and clever technique, but it is not suitable here. First, it will not work on Windows. Secondly, it would require us to stuff all knowledge of all launchers into the main java binary. This solution, in contrast, distributes this knowledge out to every individual tool, making it possible to add new tools without modifying the main static binary.

[mlbridge]

Webrevs

• 00: Full (289205ff)

[AlanBateman]

Right now, the static-jdk image is a bit strange in that it's a modular run-time image but with all native code compiled into bin/java. In time we hope that jlink will be able to create a static image where everything is in the single executable, so no bin directory (or any directory) for launchers. I don't object to doing something to help testing the intermediate step, just not sure that main line is the right place for this.

As regards the shim when I wonder if it should use CreateProcessW but maybe it doesn't matter for the test environments where they will run.

[magicus]

The way I see it is that we need to be able to produce a generic static JDK image, to be able to test it and keep it from bit-rotting. In actual practice, though, a static JDK is of limited (or rather no) use by itself, and the static build will only serve to produce component that are consumed by downstream setups, where these static libraries are incorporated more tightly into a product.

So while we continue to hammer out how to improve this, I think it is important to be able to test static builds in mainline, or they will break. I think we are going to have to keep these relaunchers around for as long as we want to test the static build, even if no downstream consumer really cares. If that is going to be kept in the "static-jdk" image, or in another deliverable, I can't say. Possibly the entire "static-jdk" will go away when we are done with proper jlink/jmod integration, or maybe not. For now, I think this works fine.

[magicus]

As regards the shim when I wonder if it should use CreateProcessW but maybe it doesn't matter for the test environments where they will run.

I must admit that I am not very well versed in Windows programming. What is the difference? I thought it was just about the encoding of the function argument. If we change to use CreateProcessW, does not all argument processing have to be changed as well, to be able to propagate command line options?

[jianglizhou]

Right now, the static-jdk image is a bit strange in that it's a modular run-time image but with all native code compiled into bin/java. In time we hope that jlink will be able to create a static image where everything is in the single executable, so no bin directory (or any directory) for launchers.

@AlanBateman Related to the topic, I just submitted openjdk/leyden@56a9e59 with experimental change for jlinking hermetic image using a pre-linked static launcher executable (when @slowhog's openjdk/leyden#46 is in place, I'll experiment with collecting the static libs from .jmod and performing native linking). The experimental change currently still writes out a complete jdk image but only the lib/modules (the hermetic executable) is needed for runtime execution (e.g. for running simple HelloWorld).

[jianglizhou]

So while we continue to hammer out how to improve this, I think it is important to be able to test static builds in mainline, or they will break.

Agree with @magicus on the importance of being able to test static builds in mainline. It would be good to include teir1 testing for the static builds on linux-x64 in GHA. Currently, we do static build on linux-x64 in GHA without running any tests.

[bridgekeeper]

@magicus This pull request has been inactive for more than 8 weeks and will be automatically closed if another 8 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[magicus]

/keepalive

[openjdk]

@magicus The pull request is being re-evaluated and the inactivity timeout has been reset.

[openjdk]

@magicus this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout static-relaunchers
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push