Support for Publishing "Slim" Packages

[james-crowley]

I wanted to kick start a discussion of adding support for slim packages. Currently, on the unofficial AdoptOpenJDK Docker images we are producing slim builds for JDKs and hopefully soon we will also publish slim JRE builds as well.

Slim builds are currently being produced by running this script, slim-java.sh. In running the script we are modifying the builds we get from adoptopenjdk.net. From my understand, this modification is what will block us from publish these slim builds into the official Docker images.

You can see more information given by @dinogun about the why this modification will be blocked here, AdoptOpenJDK/openjdk-docker#125 (comment).

To get these slim builds in the official Docker images, we would need AdopOpenJDK to build and publish these slims packages.

[aahlenst]

Serious question: What makes the same script running on the same Jenkins in a different job less official? And what is the exact definition of bar that needs to be cleared to make things "official"?

If AdoptOpenJDK declares something to be official and supported, this should be sufficient for any other project. Otherwise, things can get out of hand pretty quickly.

The "slim target" is Docker-specific. It does not make sense to reuse it for Linux packages, for example. And end-users are supposed to use the JRE or a jlinked image.

[james-crowley]

@aahlenst the bar that needs to be cleared is set by the maintainers of the Official Image repo in the Docker Library organization.

In particular the issues raised in this PR, docker-library/official-images#5710, need to be addressed. The comments from @tianon shine some light on what their concerns were with slim images. See comments, docker-library/official-images#5710 (comment), docker-library/official-images#5710 (comment) and docker-library/official-images#5710 (comment).

I think there is some discrepancies with what the maintainers of the Official Image repo wanted as the discussion went on as more facts got introduced.

At first it looked like they wanted slim builds to be distributed by AdoptOpenJDK and given their stamp of approval. But as the conversation moved on, they mentioned:

The plan at AdoptOpenJDK is to make the slim script as part of the build itself (and generate slim builds).

Honestly, if that's the case I'd prefer to add slim variants until that happens -- it's quite a bit of additional complexity that I think makes a ton more sense to have in AdoptOpenJDK itself (and thus have more eyes reviewing, maintaining, and ultimately supporting).

@tianon you seemed like the main reviewer for the PR, could you clarify what we need from Adopt to get the slim builds in the official images?

[tianon]

My biggest concern with the script that's specific-to-the-Docker-images vs being part of an official "AdoptOpenJDK build process" is in the testing of it.

My understanding is that official AdoptOpenJDK builds undergo testing that's at least in spirit similar to something like the TCK, and the Docker images most certainly do not (we do some basic surface level testing, but nothing more advanced than "does javac work?" and "does this basic program run?").

If these are going to be officially-released artifacts of the AdoptOpenJDK project, then IMO it seems prudent for them to actually be officially-released artifacts on the same level as any others, undergoing the same testing and scrutiny as any others.

If the AdoptOpenJDK project doesn't agree that the resulting "slim" artifacts should be released as such, then perhaps the minimization script itself could be part of the tarballs that currently are distributed? I think there's a lot of value in having a standard way to "slimify" a given release, and I don't think there's anything inherently "Docker-specific" about doing so.

In that way, they would be part of the AdoptOpenJDK review/release process instead of being a Docker-specific script which I think would end up only being lightly reviewed (I'm certainly not an expert in which bits of any given AdoptOpenJDK release are reasonably "safe" to trim for a slimmer build, so my review of those scripts as part of the official images "dockerization" review is not going to be particularly helpful, and I believe I can also speak for @yosifkit in that regard).

[tianon]

To put things another way, if "end-users are supposed to use the JRE or a jlinked image," I don't see what makes Docker fundamentally different? What makes the Slim image more interesting to Docker users than non-Docker users? Why aren't Docker users encouraged to use the same JRE / jlink methods as non-Docker users?

[james-crowley]

@tianon Thanks for explaining and layout what is needed/wanted.

My biggest concern with the script that's specific-to-the-Docker-images vs being part of an official "AdoptOpenJDK build process" is in the testing of it.

My understanding is that official AdoptOpenJDK builds undergo testing that's at least in spirit similar to something like the TCK, and the Docker images most certainly do not (we do some basic surface level testing, but nothing more advanced than "does javac work?" and "does this basic program run?").

The currently plan is all nightly images, which includes slim images, produced by AdoptOpenJDK will go through a certain set of tests. You can see some of the progress of connecting the AdoptOpenJDK Docker Build pipeline and the AdoptOpenJDK Test pipeline, AdoptOpenJDK/openjdk-docker#328. Myself, @smlambert, and a couple others are working on connecting these pipelines.

@smlambert could you elaborate on the set of tasks we will be running and give @tianon a little background on what these tests should be verifying?

As for the questions on packaging and officially-released artifacts, the build team can hopefully address some of the questions. @karianna @sxa

[aahlenst]

@tianon Requiring something to be fully tested and requiring some sort of standalone release are two separate things. Having the slim images tested is something that I endorse.

If I read the script correctly, the jmods are removed. In turn, jlink becomes non-functional. This might make sense for people that need more than a JRE for their applications in Docker, for example because they want to use OpenJFX or because they know they need javac but do not need jlink. Putting a partially broken JDK on the front page is something that I do not want, especially because this is very hard to communicate, a non-standard thing and regular users are expected to use jlink. That’s why Jigsaw (the JPMS) was forced upon the ecosystem, after all. Size is at a premium in a container image, but not on a regular computer, and because container images are harder to create and host, I can see the value in the slim image. Besides, I‘m not sure if it would pass the full test suite without jmods.

[smlambert]

re: #1777 (comment), initially we had planned to run some system tests, but given there are slim images included in the set, it may be the wiser course of action to focus on a subset of the functional/openjdk tests. This would allow us to verify functional correctness of the core set of APIs, and not worry about varying the set of tests depending on the image under test.

As we see how those tests do, we can increase the level of testing for the full images, if we decide it is important to do so, or audit particular pieces of functionality based on support issues that arise.

[aahlenst]

The objective of this issue is not sufficiently clear and as it stands, it isn't actionable.

• If this is about testing of the Docker images, especially of the slim variants, please create an issue in https://github.com/AdoptOpenJDK/openjdk-docker/. Test coverage of downstream artifacts needs to be dealt with downstream.
• If this is a genuine request to add "slim" binary builds of the JDK, please demonstrate that such a variant is useful on its own to our end users (not Docker-specific) and warrants the additional burden (resources, support) on the project. If this information is provided, I'm happy to reopen the issue.