[GR-56601] Add support for JCMD and the Attach API

[roberttoyonaga]

Summary

This pull request adds initial support for the Attach-API and JCMD. Specifically, other processes may now use the "attach" mechanism to communicate with Native Images using JCMD. This has been implemented as an optional feature that is excluded from the image by default (similar to JFR or heap dumping).

For now the following JCMD command types are supported:

• GC.heap_dump
• JFR.start
• JFR.stop
• JFR.check
• JFR.dump
• Thread.dump_stacks
• Thread.dump_to_file
• VM.native_memory
• VM.runtime_compilation
• help

Support for more commands should be straightforward to add in the future.

Related issue: #8915

Motivation

Native Image has a growing number of features such as NMT, heap dumps, thread dumps, and compilation dumps that rely on handling Unix signals to perform operations requested at runtime. This is not ideal because there is a limited number of Unix signals that are available to be used. It is also not ideal because arguments cannot be easily used with signals. Supporting JCMD would be a better alternative to signals because we can support an unlimited number of commands and also accept arguments.

JFR in OpenJDK heavily uses JCMD to accept user requests at runtime. Currently, there is no way to stop/start/dump recordings spontaneously at runtime (apart from using the remote JMX feature - which has significant overhead with respect to image size). This is inconvenient if a user wants to diagnose a problem without shutting down or restarting their application. For example, if a user wants to access JFR recordings from their application, they need to terminate their application so that a JFR dump is created. With JCMD they are able to dump recordings whenever they desire without needing to restart their entire application.

Usage

1. Build this PR with mx build

2. Build your executable with theattach monitoring option. This will automatically include JCMD as well. Make sure your application runs long enough for you to enter some requests through JCMD.
   mx native-image --enable-monitoring=nmt,jfr,jvmstat,threaddump,attach YourApp

3. Run your app. Then find its PID.
   ./yourapp

4. In another terminal:
   $JAVA_HOME/bin/jcmd <your app's PID> help
$JAVA_HOME/bin/jcmd <your app's PID> help JFR.stop
$JAVA_HOME/bin/jcmd <your app's PID> help help
$JAVA_HOME/bin/jcmd <your app's PID> VM.native_memory summary

Limitations

Attach-API only works on posix so far. This is not a regression since headp dump, thread dump, compilation dump, nmt dump, and JFR, all don't work on Windows anyway (since they require unix signals to operate).

The Windows implementation will be largely independent of code in this posix implementation. So there will be no need to go back and rework anything here. It would just be additional.

Other notes

• JfrArgumentParser.java is mostly just code factored out of JfrManager so that it can be reused by the JFR DCMDs.
• DumpThreadStacksSupport.java is mostly just code factored out from DumpThreadStacksOnSignalFeature.java
• Error handling in JfrManager has been changed so that we don't throw RuntimeExceptions when handling DCMDs with invalid input. Instead we just throw DcmdExceptions and fail gracefully. Avoid bad input crashing the Attach server.

[roberttoyonaga]

The GHA GraalVM Gate / /sulong style,fullbuild,sulongBasic JDK21 (pull_request) is failing due to some Truffle copyright headers failing the style check. This seems unrelated to the code in this PR.

[christianhaeubl]

Thanks for the PR, I started reviewing and integrating it so please don't push any further changes for now.

[christianhaeubl]

Is this exception really needed? As far as I can see, HotSpot often just throws an IllegalArgumentException.

[roberttoyonaga]

Yes I agree, IllegalArgumentException is probably good enough here

[christianhaeubl]

Can diagnostic commands also be triggered via some other way than the Attach API? If not, then I will merge the attach and dcmd packages.

[roberttoyonaga]

In OpenJDK, diagnostic commands can also be used through JMX (DiagnosticCommandMBean). I think they can also be used directly from within the JVM. I'm not planning to implement DiagnosticCommandMBean any time soon. And I don't really see too much value in using DCMD directly from within SVM. So it's probably fine to merge attach and dcmd packages.

[christianhaeubl]

Why can't we use jdk.jfr.internal.dcmd.DCmdDump instead?

[roberttoyonaga]

Maybe we could. But jdk.jfr.internal.dcmd.DCmdDump expects to be provided with a jdk.jfr.internal.dcmd.ArgumentParser and also writes back results differently (reportOperationComplete). In OpenJDK, each individual DCMD is responsible for sending back their own specific responses. In this PR, only the AttachListener classes know about the sockets and the DCMDs just return a simple string for the AttachListener to deal with.