OpenJ9 - "Could not create the Java Virtual Machine" when some special characters are on the command line

[bobswift]

Special characters that cause problems: — – … テ

Fails:

.../java …
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Using:

openjdk 11.0.3 2019-04-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.3+7)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.14.0, JRE 11 Linux amd64-64-Bit Compressed References 20190417_202 (JIT enabled, AOT enabled)
OpenJ9   - bad1d4d06
OMR      - 4a4278e6
JCL      - 5cc996a803 based on jdk-11.0.3+7)

Works as expected:

.../java …
Error: Could not find or load main class ???
Caused by: java.lang.ClassNotFoundException: ???

Using:

openjdk 11.0.3 2019-04-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.3+7)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.3+7, mixed mode)

The above is just a quick way to show the error. The real case is trying to pass special characters in parameters to a jar entry point ;). This also worked correctly on standard JDK 8.

[DanHeidinga]

Mirrored this issue to the OpenJ9 repo - eclipse-openj9/openj9#5751

[pdbain-ibm]

@bobswift
Thank you for reporting this.

As I understand the issue, you are passing a special character, such as ellipsis (…) as the sole argument to the Java launcher, which it should interpret as the main class name, correct? Further, you expect the JVM to report this special character as missing class, right?

I ran your test case under the bash shell on Ubuntu 16 using the Java 11 build you used (and others) and I get the expected behaviour:

$ java -version
openjdk version "11.0.3" 2019-04-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.3+7)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.14.0, JRE 11 Linux amd64-64-Bit Compressed References 20190417_202 (JIT enabled, AOT enabled)
OpenJ9   - bad1d4d06
OMR      - 4a4278e6
JCL      - 5cc996a803 based on jdk-11.0.3+7)
$ java …
Error: Could not find or load main class …
Caused by: java.lang.ClassNotFoundException: …

I tested with the other special characters you mentioned and obtain similar results.

1. Am I running your test case correctly?
2. Are the results I obtain the same as the once you believe to be correct?
3. Would you please re-test and verify that you are still seeing the issue?
4. If so in 3., please re-run with -verbose:init and send me the result.
5. If this test is derived from a more complicated test, can you share the latter?

If you want to discuss this directly, please contact me on Slack at openj9.slack.com.

Thank you
-p

[bobswift]

Thanks for the info.
The real use case is calling a java program (in a jar) with parameters containing special characters. However, this was the easiest way to reproduce without any code dependencies.

1. Your scenario looks the same
2. Yes, that is what I would expect
3. Thanks for the logging tip, here is the log. This is running from an alpine based docker image:

bash-4.4$ $JAVA11_HOME/bin/java -verbose:init  …
Adding command line arguments
VM known paths  - j9libvm directory: /sde/java11/lib/compressedrefs
                - j2seRoot directory: /sde/java11/lib/compressedrefs
Option 0 optionString="-Xoptionsfile=/sde/java11/lib/options.default" extraInfo=(nil) from environment variable ="N/A"
Option 1 optionString="-Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple" extraInfo=(nil) from environment variable ="N/A"
Option 2 optionString="-Xjcl:jclse29" extraInfo=(nil) from environment variable ="N/A"
Option 3 optionString="-Dcom.ibm.oti.vm.bootstrap.library.path=/sde/java11/lib/compressedrefs:/sde/java11/lib" extraInfo=(nil) from environment variable ="N/A"
Option 4 optionString="-Dsun.boot.library.path=/sde/java11/lib/compressedrefs:/sde/java11/lib" extraInfo=(nil) from environment variable ="N/A"
Option 5 optionString="-Djava.library.path=/sde/java11/lib/compressedrefs:/sde/java11/lib:/usr/lib64:/usr/lib" extraInfo=(nil) from environment variable ="N/A"
Option 6 optionString="-Djava.home=/sde/java11" extraInfo=(nil) from environment variable ="N/A"
Option 7 optionString="-Duser.dir=/var/atlassian/bamboo" extraInfo=(nil) from environment variable ="N/A"
Option 8 optionString="-verbose:init" extraInfo=(nil) from environment variable ="N/A"
Option 9 optionString="-Djava.class.path=." extraInfo=(nil) from environment variable ="N/A"
Option 10 optionString="-Dsun.java.command=…" extraInfo=(nil) from environment variable ="N/A"
Option 11 optionString="-Dsun.java.launcher=SUN_STANDARD" extraInfo=(nil) from environment variable ="N/A"
Option 12 optionString="-Dsun.java.launcher.pid=30869" extraInfo=(nil) from environment variable ="N/A"
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Sorry, not a member of the slack group.

[pdbain-ibm]

@bobswift I would encourage you to join the Slack group. You can request an invitation via this page:
https://www.eclipse.org/openj9/index.html#contributing: click the "(Request an invite)" link.

This may be a Docker issue, since it seems to run fine in a bare Linux shell. Are you able to test in a non-Docker environment? I will run your test case in a Docker container.

[pdbain-ibm]

I tried it in a docker image built from https://github.com/eclipse/openj9/tree/master/buildenv/docker/jdk8/x86_64/ubuntu16 and I get the expected (correct) behaviour.

[pdbain-ibm]

Status copied from Slack channel:
re: #1080
I can confirm that it is working correctly on the linux that we host our docker containers on:
Linux nuc 4.15.0-24-generic #26-Ubuntu SMP Wed Jun 13 08:44:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Doing some other experiments.

Peter Bain [4:00 PM]
Thanks for the feedback. I assume java HelloWorld works properly in your container?
Bob Swift [4:06 PM]
Yes, hundreds of tests run just fine. But everyone that had a parameter containing a special character failed the same way.
I have tested with the base container our container is based on and that worked fine. Tested on another instance of our container and it had the same problem. Not sure what to try next.
This works: java HelloWorld
This fails: java HelloWorld テ (edited)

[pdbain-ibm]

@bobswift

This fails: java HelloWorld

That is very interesting: it's not the main class, but any argument.
There are a few things to try:

1. Test with a fresh copy of the JDK in a directory local to the container. Please try with an absolute path to java. I suggest trying a different Java version, e.g. Java 8.
2. Run with tracing enabled, by adding the following JVM argument: -Xtrace:maximal=all,output=trace. This will create a file called trace (you can use a different name if you wish). Please send me that file.
3. I can try the test locally if you can provide the files and instructions on building the container.

If you have any questions, contact me on Slack.

Thanks again for your help.
-p

[pdbain-ibm]

@bobswift
Have you been able to retest per #1080 (comment)?

[pdbain-ibm]

@DanHeidinga I suggest closing this unless @bobswift has further input.

[DanHeidinga]

I don't have rights to close it on this repo. @sxa555 Can you close this?

[bobswift]

Sorry, didn't have time to do anymore scenarios. We had to stop using OpenJ9. If I have time in the future to try again I will open another issue.

[pdbain-ibm]

@bobswift Please let us know if there is anything we can do to facilitate you using OpenJ9.

[usernameak]

Bug still exists. Reopen. Try running jar with unicode in path on Windows and you'll get that error.
I use jdk8u222-b10_openj9-0.15.1.

[JasonFengJ9]

Try running jar with unicode in path on Windows and you'll get that error. I use jdk8u222-b10_openj9-0.15.1.

Does same error happen to hotspot builds?
What's the machine environment, OS version, locale settings, etc.?

[usernameak]

Does same error happen to hotspot builds?

no

What's the machine environment, OS version, locale settings, etc.?

Windows 7 Professional x64
Russian locale
However, it happened on another machine with Win10 too