[GR-61548] Add documentation and codeowners for the SubstrateVM JDWP implementation

docs/reference-manual/native-image/DebuggingAndDiagnostics.md

@@ -16,4 +16,5 @@ Native Image provides utilities for debugging and inspecting the produced binary
  - For performance analysis, see [Linux Perf Profiler Support in Native Image](PerfProfiling.md)
  - For an overall insight regarding build phases and the contents of a native executable, use [Build Reports](BuildReport.md)
  - For native memory tracking, see [Native Memory Tracking (NMT)](NMT.md)
- - See the [Java Diagnostic Command documentation](JCmd.md) for instructions on using `jcmd`.
+ - See the [Java Diagnostic Command documentation](JCmd.md) for instructions on using `jcmd`.
+ - For Java Debug Wire Protocol (JDWP) support in Native Image to enable debugging with standard Java tooling, see [Java Debug Wire Protocol (JDWP)](JDWP.md).

docs/reference-manual/native-image/JDWP.md

@@ -0,0 +1,153 @@
+---
+layout: docs
+toc_group: debugging-and-diagnostics
+link_title: Java Debug Wire Protocol (JDWP) with Native Image
+permalink: /reference-manual/native-image/debugging-and-diagnostics/JDWP/
+---
+
+# Java Debug Wire Protocol (JDWP) with Native Image
+
+## Overview
+
+This document describes the Java Debug Wire Protocol (JDWP) debugging support for Native Image, a feature that enables debugging of native images using standard Java tooling.
+
+## Installation
+
+The JDWP feature is not included by default as part of Native Image. To use it, you need to [build GraalVM from source](https://github.com/oracle/graal/blob/master/vm/README.md) using the following [mx](https://github.com/graalvm/mx/) commands:
+```shell
+mx --dynamicimports /substratevm build
+export JAVA_HOME=$(mx --dynamicimports /substratevm graalvm-home)
+```
+
+## Usage
+
+> Note: JDWP debugging for Native Image is currently under development.
+
+To include JDWP support in a native image, add the `--macro:svmjdwp` option to your `native-image` command:
+
+```shell
+native-image --macro:svmjdwp ... -cp <class/path> YourApplication ...
+```
+
+This command produces:
+1. The native executable
+2. An `<image-name>.metadata` file
+3. The `lib:svmjdwp` (`libsvmjdwp.so`, `libsvmjdwp.dylib` or `svmjdwp.dll`) shared library that will be necessary when debugging is also copied next to those files.
+
+### Launching in Debug Mode
+
+To launch the native image in debug mode, use the `-XX:JDWPOptions=` option, similar to HotSpot's `-agentlib:jdwp=`:
+
+```shell
+./your-application -XX:JDWPOptions=transport=dt_socket,server=y,address=8000
+```
+
+> Note: Debugging requires the _image-name.metadata_ file generated at build time and the `svmjdwp` shared library in the same directory as the native executable.  
+
+For a complete list of supported JDWP options on Native Image, run:
+
+```shell
+./your-application -XX:JDWPOptions=help
+```
+
+### Additional JDWP Options
+
+Native Image supports additional non-standard JDWP options:
+
+- `mode=native:<path>`: Specifies the path to the `svmjdwp` library. This can be:
+  - A direct path to `lib:svmjdwp`
+  - A directory containing `lib:svmjdwp`
+  - A GraalVM installation containing `lib:svmjdwp` in the `lib` or `bin` directory
+
+   If no path is specified, `lib:svmjdwp` is searched for beside the native executable.
+
+Examples:
+- `-XX:JDWPOptions=...,mode=native:<path/to/lib:svmjdwp>`
+- `-XX:JDWPOptions=...,mode=native:<path/to/directory/containing/lib:svmjdwp>`
+- `-XX:JDWPOptions=...,mode=native:<path/to/java/home>`: Search `lib:svmjdwp` inside `JAVA_HOME`, for example `lib|bin/lib:svmjdwp`.
+- `-XX:JDWPOptions=...,mode=native`: Search `lib:svmjdwp` besides the native executable directory.
+
+- `-XX:JDWPOptions=...,vm.options=...`: VM options, separated by whitespaces, passed to the JDWP server isolate/JVM, should not include a `,` character.
+- `-XX:JDWPOptions=...,vm.options=@argfile`: Also supports [Java Command-Line Argument Files](https://docs.oracle.com/en/java/javase/21/docs/specs/man/java.html#java-command-line-argument-files).
+
+Note: If `lib:svmjdwp` cannot be found, the application will terminate with error code 1.
+
+### Build `lib:svmjdwp`
+
+Run `mx --native-images=lib:svmjdwp build` to build the library.
+
+### Build a Native Executable with JDWP Support
+
+Add the `--macro:svmjdwp` option to the `native-image` command:
+```shell
+mx native-image --macro:svmjdwp -cp <class/path> MainClass ...
+```
+
+To build and include `lib:svmjdwp` as a build artifact, run:
+```shell
+mx --native-images=lib:svmjdwp native-image --macro:svmjdwp -cp <class/path> MainClass ...
+```
+
+Both commands produce a binary, an `<image-name>.metadata` file.
+
+## Goals and Constraints
+
+The JDWP debugging support for Native Image aims to:
+
+1. Expose Native Image through JDWP as-is, maintaining its assumptions and constraints
+2. Incur minimal or no performance overhead when not in use
+3. Add minimal size overhead to the native binary
+4. Be available on all Graal-supported platforms, including Linux, macOS, and Windows, across x64 and AArch64 architectures
+5. Provide a debugging experience similar to HotSpot, without requiring additional steps (e.g., setting permissions, environment variables)
+
+## Architecture
+
+The JDWP debugging support is implemented using a Java bytecode interpreter, adapted from [Espresso](https://github.com/oracle/graal/tree/master/espresso) to work with Native Image. Key components include:
+
+1. **Interpreter**: Derived from Espresso and adapted for [SubstrateVM](https://github.com/oracle/graal/tree/master/substratevm/). It does not enable any dynamic features beyond what Native Image already supports.
+
+2. **PLT/GOT Feature**: Used to divert execution to the interpreter. This implementation detail may change for some platforms.
+
+3. **Metadata File**: An external _.metadata_ file produced at build time, containing information required for runtime method interpretation.
+
+4. **JDWP Server**: Implemented as a native library (`lib:svmjdwp`), handling network connections and implementing JDWP commands.
+
+5. **JDWP Resident**: A component within the application providing access to locals, fields, stack traces, and other runtime information.
+
+## Limitations
+
+The JDWP debugger for Native Image is designed to align with Native Image's architecture and principles.
+While many limitations are a natural consequence of Native Image's design, others may be due to the current implementation of the debugger itself.
+Here are the key limitations to be aware of:
+
+- The debugger follows Native Image closed-world assumptions:
+  - Only classes, methods, and fields included in the image are accessible.
+  - Some types may not be instantiable at runtime, even if there are instances in the image heap.
+  - Some fields cannot be written to.
+  - No support for dynamic class loading.
+  - No class or method redefinition.
+  - There's no runtime class-path `System.getProperty("java.class.path") == null`
+- No exception breakpoints.
+- No field watchpoints.
+- No early return or frame popping.
+- Some methods cannot be interpreted by the debugger (see below):
+  - Methods that use "System Java".
+  - Methods that contain a call to an intrinsic without compiled entry-point.
+  - Breakpoints cannot be set in non-interpretable methods.
+  - Stepping through non-interpretable methods is not possible, these are effectively treated as if they were Java "native" methods, with no guarantee to break/step on the next executed method, only on the next interpreted method.
+- Not all execution paths are executable/interpretable.
+  - Interpreting "dead-code" may work, but only on a best-effort basis.
+  - Violating compiled-code assumptions, for example, passing a null argument where a non-null was expected, is considered undefined behavior and prone to crashes.
+- Cannot write locals of compiled frames.
+- Cannot hit breakpoints or stepping events on actively executing compiled methods.
+- Step-out operations only work for interpreter frames, not compiled frames.
+- Can only debug the first isolate of a native image.
+- Step-into does not work for target methods of a `MethodHandle` object, for example, lambdas.
+
+These limitations reflect the current state of JDWP debugging support in Native Image.
+Some may be addressed in future iterations of the debugger, while others are fundamental to Native Image's design.
+
+### Further Reading
+
+- Instructions on how to run the JDWP server in HotSpot, so that the debugger can be debugged: [JDWP server](https://github.com/oracle/graal/tree/master/substratevm/src/com.oracle.svm.jdwp.server/README.md)
+- Implementation details about the interpreter and the transitions from and to compiled code: [SubstrateVM Interpreter](https://github.com/oracle/graal/tree/master/substratevm/src/com.oracle.svm.interpreter/README.md)

substratevm/src/com.oracle.svm.core.graal.aarch64/OWNERS.toml

@@ -0,0 +1,5 @@
+[[rule]]
+files = "*"
+all = [
+    "bernhard.urban-forster@oracle.com",
+]

substratevm/src/com.oracle.svm.core.graal.amd64/OWNERS.toml

@@ -0,0 +1,5 @@
+[[rule]]
+files = "AMD64InterpreterStubs.java"
+all = [
+    "bernhard.urban-forster@oracle.com",
+]

substratevm/src/com.oracle.svm.interpreter.metadata/OWNERS.toml

@@ -0,0 +1,12 @@
+[[rule]]
+files = "*"
+all = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]
+any = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]

substratevm/src/com.oracle.svm.interpreter/OWNERS.toml

@@ -0,0 +1,12 @@
+[[rule]]
+files = "*"
+all = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]
+any = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]

substratevm/src/com.oracle.svm.interpreter/README.md

@@ -0,0 +1,168 @@
+# Java bytecode interpreter for SubstrateVM
+
+## How execution is diverged from compiled code to interpreter
+
+Execution of a native-image binary will start out in AOT compiled code as usual.
+Execution of specific methods can be diverted into the interpreter via `InterpreterDirectives`.
+
+There are two cases to consider:
+
+1. There is an activation frame on a thread stack: We have to deoptimize that frame so that whenever execution returns to this activation frame it will transfer execution to the interpreter.
+2. Whenever a call to this method is done, we need to make sure that its execution will happen in the interpreter.
+
+
+For 1. the procedure is similar to the `Deoptimizer` implemented for run-time compiled methods: We patch the return address in the upper frame, so that it points to a deopt stub that will do the deoptimization of the target frame. Once the deopt stub executes it collects the locals and arguments on the native frame accordingly, and tail calls into a small stub that will create a new frame and resumes execution in the interpreter with the right `bci` and locals in place.
+
+For 2. we piggyback on the PLT+GOT work done in the context of code compression (see GR-40476).  That means every call has an extra indirection through a PLT stub.  There are other approaches that work on some platforms (e.g., map `.text` section `rwx` on Linux and patch the prologue or call-sites accordingly), that will be reconsidered once interpretation is enabled by default.
+
+### Transition from compiled code to interpreter (`c2i`)
+
+If we want a method to be executed in the interpreter, we can update its slot in the GOT accordingly, since each call-site for that method will go through the PLT+GOT mechanism. But _what_ do we patch there? For each method that we want to run in the interpreter we allocate an `InterpreterMethod` at image build-time and store it in a table. We then create a `interp_enter_trampoline` that injects its index to the `InterpreterMethod` as a hidden argument.
+
+```c
+interp_enter_trampoline:
+	mov $interp_reg, <index to InterpreterMethod>
+	tailcall interp_enter_stub
+```
+
+The `interp_enter_stub` then collects all ABI registers into a `InterpreterEnterData` stackvalue:
+
+```c
+struct InterpreterEnterData {
+	InterpreterMethod method;
+	Pointer stack;
+	Word abi_arg_gp_regs[];
+	Word abi_arg_fp_regs[];
+	Word abi_ret_gp_reg;
+	Word abi_ret_fp_reg;
+}
+
+interp_enter_stub:
+	InterpreterEnterData enterData = stack_allocate();
+	enterData->stack = $original_sp;
+	enterData->method = interpreter_methods_table[$interp_reg]
+
+	for (int i = 0; enterData->abi_arg_gp_regs.length; i++) {
+		 enterData->abi_arg_gp_regs[i] = getABIRegGP(i);
+	}
+	for (int i = 0; enterData->abi_arg_fp_regs.length; i++) {
+		 enterData->abi_arg_fp_regs[i] = getABIRegFP(i);
+	}
+
+	mov arg_0_reg, enterData
+	call interp_enter
+
+	mov return_gp_reg, enterData.abi_ret_gp_reg
+	mov return_fp_reg, enterData.abi_ret_fp_reg
+	ret
+```
+
+And then `interp_enter` allocates the actual `InterpreterFrame`, populates it with arguments and calls the interpreter main loop:
+
+```java
+static void interp_enter(InterpreterEnterData enterData) {
+	InterpreterMethod method = enterData.method;
+	InterpreterFrame frame = allocate();
+
+	CallInfo cinfo = method.getCallInfo();
+	for (int i = 0; i < method.signature.length; i++) {
+		// cinfo.getArg does the "heavy" ABI work
+		frame.setArg(i, cinfo.getArg(i, method.signature[i], enterData));
+	}
+
+	new Interpreter(method).execute(frame);
+
+	enterData.setReturnValue(cinfo.getReturnValue(frame));
+}
+```
+
+Note that:
+* Per method there is one `interp_enter_trampoline`, and it is architecture specific.
+* There is one `interp_enter_stub` globally, and it is architecture specific.
+* There is one `interp_enter` globally, and is written in System Java.
+* `InterpreterEnterData` is a `CStruct`.
+
+### Transition from interpreter to compiled code (`i2c`)
+
+Similar approach as for `c2i`: We have `interp_exit` that builds a `InterpreterExitData` struct, which is then passed down to an `interp_exit_stub` that does the ABI related work. Eventually it calls the AOT compiled method indirectly via its GOT offset (i.e. we do _not_ go through the PLT stub) and makes sure the return values are correctly passed up.
+
+For exit we do not require a trampoline per method.  However the `interp_exit_stub` is a bit special, as it does not have a fixed frame layout, something the Graal backend cannot deal with well. Therefore there is special handling around this, for example the stack walker needs to know how much it should unwind.  The variable length of such frames is stored at a known offset.
+
+### Call dispatch
+
+Let's consider multiple scenarios.
+
+#### Scenario A: Static call, no inlining
+
+Consider:
+```java
+class Calls {
+	static void foo() {
+		bar();
+	}
+	static void bar() {
+		print("bar");
+	}
+}
+```
+
+Cases:
+1. If we want `Calls#bar` to be executed in the interpreter, its GOT slot will be patched to an `interp_enter_trampoline` that injects the `InterpreterMethod` for `Calls#bar`.
+2. If `Calls#foo` is executed in the interpreter, and `Calls#bar` should run as compiled code, we will do a GOT dispatch with the `i2c` primitive described above. A prerequisite for that is that we can determine the right GOT slot at the time we create the constant pool entry for that `invokestatic` in `Calls#foo`, which is effectively a `MethodPointer` attached to the `InterpreterMethod`.
+3. If both should be executed in the interpreter we can either go through two transitions via `i2c <> c2i`.  There is some optimization potential here by having an `i2i` transition.
+
+
+#### Scenario B: virtual call, no inlining
+
+Consider:
+```java
+class VCalls {
+	abstract void baz();
+
+	static void dispatch(VCalls o) {
+		o.baz();
+	}
+}
+
+class VCalls_A extends VCalls {
+	void baz() {
+		print("VCalls_A");
+	}
+}
+
+class VCalls_B extends VCalls {
+	void baz() {
+		print("VCalls_B");
+	}
+}
+```
+
+Cases:
+1. Assume `VCalls#dispatch` runs as compiled code and  `VCalls_A#baz` should run with the interpreter. Since the virtual dispatch goes through the PLT+GOT mechanism, we can patch the GOT slot accordingly.
+2. Assume `VCalls#dispatch` runs in the interpreter and both `VCalls_A#baz` and `VCalls_B#baz` run as compiled code. We do the virtual dispatch by access the entrypoint via `o->hub->vtable[vtableIndex]`, and then pass that through the `i2c` mechanism above. Additionally, at build-time we need to attach the right `vtableIndex` for the constant pool entry associated with that `invokevirtual` instruction.
+3. Assume `VCalls#dispatch` runs in the interpreter, `VCalls_A#baz` runs in the interpreter too but `VCalls_B#baz` runs as compiled code. In this case we do the same as in 2. and accept the overhead of a `i2c <> c2i` transition.
+
+#### Scenario C: Static call, with inlining
+
+Consider:
+```java
+class Calls {
+	static void foo() {
+		bar();
+	}
+	static void bar() {
+		print("bar");
+	}
+}
+```
+
+But this time `Calls#bar` is inlined. Cases:
+1. Assume `Calls#bar` should be executed in the interpreter. In this case we need to make sure that the execution of `Calls#foo` already happens in the interpreter, i.e. patch its GOT slot and deoptimize activation frames of it.
+2. Assume `Calls#foo` should be executed in the interpreter. As soon as we reach the `invokestatic` to call `Calls#bar` we cannot do the same as in Scenario A, because there is no AOT compiled version for `Calls#bar` (the only usage was inlined). So we have to make sure to include a `InterpreterMethod` for `Calls#bar` at build-time, and do its execution in the interpreter. Since there is no `MethodPointer` attached to the `InterpreterMethod` of `Calls#bar`, we know that we must execute `Calls#bar` in the interpreter too then.
+
+For 2. we need to track inlining decisions during image building and persist that information in a `.metadata` companion file.
+
+#### Scenario D: Virtual calls, with inlining
+
+If virtual callees "disappear" due to inlining, SVM still reserves a vtable slot for it but populates it with a bailout stub.
+We detect such cases upon invocation and fall back to a "side vtable" populated with `InterpreterMethod`s. We construct that table during image build time and include that in the `.metadata` companion file.

substratevm/src/com.oracle.svm.jdwp.bridge/OWNERS.toml

@@ -0,0 +1,12 @@
+[[rule]]
+files = "*"
+all = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]
+any = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]

substratevm/src/com.oracle.svm.jdwp.resident/OWNERS.toml

@@ -0,0 +1,12 @@
+[[rule]]
+files = "*"
+all = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]
+any = [
+    "alfonso.peterssen@oracle.com",
+    "bernhard.urban-forster@oracle.com",
+    "martin.entlicher@oracle.com",
+]