Are Java memory options still needed for docker?

[Lewiscowles1986]

I was directed to this repo in elastic/elasticsearch-docker#241

Feature Description

Remove excess Java memory options to make compatible with modern Docker + Java workflow

https://github.com/elastic/elasticsearch-docker/blob/master/.tedi/template/docker-compose.yml.j2#L53

could instead be -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap if you use supporting JVM with container support.

Alternatively -XX:+UnlockDiagnosticVMOptions promises to also adjust CPU to respect cgroup / isolation environments, and appears from docker-library/openjdk#245 that OpenJDK official docker images support the backport to Java 8

The TLDR is that this tells docker to use the container, not host config to set sane limits.

Oracle docs here https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits

Blog post https://royvanrijn.com/blog/2018/05/java-and-docker-memory-limits/

Happy to PR & run some tests if you're up for one.

[elasticmachine]

Pinging @elastic/es-core-infra

[dliappis]

Thank you for your interest in the Elasticsearch Docker image and for raising this issue with useful details and linking the necessary articles.

We talked about this in a team discussion yesterday.

While -XX:+UseContainerSupport is enabled by default on Java >=10 and thus could infer heap values based on available memory in the container (limited with --memory), we still rely on -Xms/-Xmx for a number of reasons:

1. The main motivation is that we are striving for a uniform experience for all users of Elasticsearch, regardless of artifact/packaging format and want a common approach on how to adjust heap size for Elasticsearch. This should be possible by looking at our docs, discuss forums and blog articles, i.e. setting it via jvm.options or the environment variable ES_JAVA_OPTS. If we change the approach for Docker only, we'll need to have a lot of if clauses in our documentation and forums, and every time we deal with a question we have to branch out our checklists based on whether we are running on Docker.

2. We make ergonomic choices for direct memory, in particular for MaxDirectMemorySize by inspecting -Xms/-Xmx (set in jvm.options or overriden via ES_JAVA_OPTS), so this mechanism wouldn't work anymore if we rely on heap autodetection via -XX:+UseContainerSupport.

[jasontedor]

Note that point 2. here is no longer valid since #41962 since we inspect the heap size as the JVM would see it, whether or not it is set explicitly.