Invoke debugger function evaluations through managed UCO

[jkoritzinsky]

Summary

Layer 3 of the stack for #123864, following #133658. The prior func-eval-specific proposal is #126809.

This layer started as replacing the final native managed-call dispatch with a managed UnmanagedCallersOnly entrypoint. It now moves most debugger function-evaluation preparation, invocation, storage, and copy-back into managed CoreLib while retaining the debugger-specific control flow and GC-safe bootstrap in native code.

Managed invocation and storage

• Add Debugger.FunctionEvaluation, a runtime-owned UCO entrypoint that resolves the exact target method from its declaring RuntimeType and rooted IRuntimeMethodInfo.
• Extend the shared reflection invocation emitter with debugger-specific thunks that prepare arguments, invoke methods and constructors, copy back writable arguments, and publish results.
• Emit newobj for debugger object construction instead of preallocating an object and calling its constructor as an ordinary method. Existing-object constructor calls remain supported, and abstract/interface allocation is rejected.
• Keep the entire managed call-address vector registered through GCFrameRegistration, using exact typed locals for memory-backed by-value ref structs so their managed byrefs are reported by normal JIT GC info.
• Handle object, primitive, enum, pointer, value-type, nullable, literal, memory, register, and byref argument representations in managed code while preserving native-endian storage and debugger copy-back semantics.
• Align nullable behavior with the existing debugger contract: direct value-type memory homes are mutated in place, while debugger handles are not treated as writable nullable homes. Exact register-backed nullable temporaries are boxed only after their MethodTable is instance-active.
• Move ordinary value-type result storage into temporary GC-described external memory and defer boxing until after invocation and argument copy-back. Keep byref-like results in persistent external storage and raw byref results rooted directly in the debugger result slot.
• Create result handles in managed code only after the result is fully prepared, and preserve object identity and updated field contents across collections and later debugger inspection.

Native debugger and GC integration

• Retain native method/signature validation, generic resolution, class initialization, receiver validation inputs, register/home capture, abort state, exception publication, completion, and result ownership.
• Capture contiguous register value-type snapshots before the first GC-triggering managed transition and describe them with their exact type layout.
• Root non-leaf register references and register value-type snapshots with ExternalMemoryHandle registrations so collections during managed preparation relocate their contents correctly.
• Represent external memory with a full tagged TypeHandle, allowing the runtime and cDAC to distinguish inline value types, object-reference slots, managed-byref slots, and unmanaged pointer/function-pointer locals.
• Report each external-memory type's loader allocator during GC scanning, keeping collectible layouts alive while external storage is registered.
• Add dedicated GC-safe object-register copy-back and native-endian scalar register copy-back paths.
• Remove stale native argument copies and architecture-specific packing layers that are no longer needed, while preserving the architecture-specific register-home logic required by debugger IPC.
• Treat the debugger UCO entrypoint as a runtime-internal reverse transition for exception handling, and exclude its debugger-only MethodDesc state on WebAssembly targets that do not build debug/ee.
• Update the cDAC ExternalMemoryHandles reader, descriptors, tests, and authoritative contract documentation for the exact managed-local representation.

Correctness fixes included

• Avoid double relocation of overlapping byrefs in by-value ref structs (Debugger func-eval: a by-value ref struct with overlapping ref fields is reported to the GC twice, so a compacting GC relocates the byref twice #135245).
• Keep raw byref results rooted until deferred debugger completion (Debugger func-eval byref result can become stale before deferred completion #135247).
• Prevent stale non-leaf register values after a compacting GC (Debugger func-eval can use stale non-leaf register values after GC #135249).
• Preserve current external-result fields across collections and later inspection (Debugger external-memory func-eval results can expose stale cached fields after GC #135251).
• Keep collectible loader allocators alive while external-memory layouts are scanned (Func-eval ref struct result: ExternalMemoryHandle doesn't keep a collectible LoaderAllocator alive (access violation in GC after unload) #135252).

Validation

Validation used the existing internal MDbg/xUnit debugger-test harness with a private CoreRun and ProjectK/CoreCLR-only configuration:

• Targeted nested-byref invariant under target-only GCStress=0x3: 1/1 passed.
• Revised normal matrix: 35/35 passed.
• Targeted stress matrix: 4/4 passed.
• Windows x64 and x86 Checked native, CoreLib, and ReadyToRun builds completed with zero warnings and errors.
• cDAC unit tests and generated usage/documentation checks passed for the ExternalMemoryHandle descriptor changes.

Coverage includes moving GC, overlapping and nested byrefs, memory and register homes, non-leaf registers, receiver mutation, constructors, primitive/native-endian copy-back, nullable memory homes, byref/byref-like/value-type results, collectible generics and layouts, nested and exception-time eval, safe/rude/exception-unwind aborts, and post-eval inspection. Runtime hashes and loaded modules were audited; each abort scenario completed the callback, successfully evaluated Echo(42), detached, and exited MDbg with code 100. Test configurations were restored and no debugger/test processes survived the runs.

These results were recorded on the validated pre-refresh candidate. The final stack rebase preserved the layer's net diff; builds and tests were not repeated after the history rewrite at the user's request.

The final native invocation cleanup layer is #133660.

---

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

Resolves #135245
Resolves #135247
Resolves #135249
Resolves #135251
Resolves #135252

Note

This PR description and implementation were prepared with GitHub Copilot. Commits that reuse am11's code retain the requested co-author attribution.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[am11]

discussed on #126542.

nit: funceval was already split: #126809 so #126542 discussion is irrelevant.

[jkoritzinsky]

Thanks, @am11. Corrected the summary to reference #126809 for the prior func-eval-specific proposal rather than #126542.

Note

This reply was prepared with GitHub Copilot.

[am11]

@jkoritzinsky, please ask copilot to double check this review by my instance:

---

I reviewed the managed entrypoint, the invoker emitter, the native func-eval changes, external-memory GC reporting, and the cDAC changes. The overall design holds up and CI has no failures caused by this PR, but I found two bugs and a mismatch between what the runtime and the cDAC report. Line numbers below refer to head commit 22e799b1c3e.

Bugs

1. Evals that return a struct leak their result buffer and a GC root

funceval.cpp L1639

context.ppTemporaryResult = &temporaryResult;

& on a Wrapper returns a temporary TypedAddressInitHolder. Its destructor calls Acquire() at the end of this statement, but the holder is still null at that point, so it never takes ownership (m_acquired stays FALSE). DebugDebugger_PrepareFuncEvalResult (L1223-L1225) then stores the new DebuggerExternalMemoryOwner through the raw pointer. BaseHolder::Release() only frees when m_acquired is set, so these all do nothing:

• temporaryResult = nullptr in the catch block (L1671)
• temporaryResult.Release() (L1691)
• the holder's destructor
  Nothing else frees the owner. The owner's destructor is what releases its ExternalMemoryHandle, so each leaked owner also leaves a registered GC root. The GC keeps reporting the buffer's references, and the type's loader allocator, forever.
  When it happens: any eval that returns a struct that isn't a ref struct, such as DateTime, KeyValuePair<,>, Nullable<T> or an enum, or that constructs a value type. A watch window leaks once per refresh, keeps every object referenced from the result alive, and prevents a collectible AssemblyLoadContext from unloading.
  Fix: result avoids this because &result is taken inside the InvokeManagedFuncEval(...) call (L1664), so the holder takes ownership when that call returns. temporaryResult needs the same pattern, or a plain pointer with explicit cleanup on both paths.

2. A struct receiver held in a register is now always written back, which fails in caller frames

Debugger.FunctionEvaluation.cs L486-L494 and funceval.cpp L326-L329
The old UnpackFuncEvalArguments skipped this on purpose ("Skip over the 'this' arg, since this function is not supposed to mess with it."). Now, when the declaring type is a value type and the receiver came in a register, NeedsCopyBack(0) returns true. The emitted IL then calls CopyBackArgument(0), which calls WriteFuncEvalRegister, which calls SetRegisterValue.
For a non-leaf (caller) frame the register address is kNonLeafFrameRegAddr, so SetRegisterValue throws CORDBG_E_FUNC_EVAL_CANNOT_UPDATE_REGISTER_IN_NONLEAF_FRAME. That happens after the call has already succeeded, and the eval is reported as an exception.
When it happens: select a caller frame and evaluate dt.ToString() (or any instance method on a small struct) where the JIT keeps the struct in a callee-saved register. This worked before this PR.
In the leaf frame the call now changes the debuggee's register-held struct, which is also a behavior change. It matches how memory-homed receivers already behave, so it may be intended. If it is, it should be stated, and the non-leaf case should either skip the write-back or check for it.

3. The runtime and the cDAC report different roots for overlapping byref fields

siginfo.cpp L5206 and src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/ByRefPointerOffsetsReporter.cs
ByRefPointerOffsetsReporter::Report now skips byref slots already reported by an earlier overlapping field in an explicit-layout ref struct. That is correct: reporting the same interior pointer twice relocates it twice. The legacy DAC calls the VM reporter directly, so it gets the fix.
The cDAC copy of this reporter isn't changed by this PR and has no overlap handling. It's used by both ExternalMemoryHandles_1.cs and CallingConvention_1.cs, the stack-frame argument reporting. So for something like [StructLayout(LayoutKind.Explicit)] ref struct S { [FieldOffset(0)] ref int a; [FieldOffset(0)] ref int b; }, the cDAC reports two interior roots where the runtime reports one. docs/design/datacontracts/ExternalMemoryHandles.md doesn't describe skipping duplicates either.
There's also no runtime test for the overlap case. The change affects every caller of ReportPointersFromValueType, not only external memory handles, so a test would be worthwhile.

Possible issues

4. A by-ref double in a float register may be written back to the wrong register

funceval.cpp L713

CorDebugRegister reg = widePrimitive ? pArg->argHome.reg1 : GetArgAddrFromReg(pArg);

For a float-register home (RAK_FLOAT), FloatRegValueHome::CopyToIPCEType doesn't appear to set reg1. Reads go through GetArgAddrFromReg, which uses floatIndex, but for R8/I8/U8 this write uses reg1. The old code seems to have had the same problem, but this line is new, so it's a good moment to fix it.

5. Field values of external-memory structs may read freed memory

src/coreclr/debug/di/divalue.cpp (field value creation around L3183-L3195, refresh around L2981-L2988)
Child field values get m_isExternalMemory = true and re-read target memory on each access, but they don't hold a reference to the vmExternalMemoryOwner. Once the parent value is neutered, DB_IPCE_DISPOSE_EXTERNAL_MEMORY_OWNER frees the buffer, and a surviving child value reads freed memory in the debuggee. Before, it returned a stale but consistent copy. GetFieldValueForType also refreshes without the "process must be stopped" check that GetValue now has.

6. The left side now requires typeHandle in type node 0 for register-held structs

The right side now fills ClassTypeData.typeHandle in node 0, and the left side returns CORDBG_E_CLASS_NOT_LOADED if it is missing. That's fine while mscordbi is version-locked to the runtime, but a mismatched pair would break.

Smaller points

• Unrelated fix in this PR. RuntimeTypeHandle::InternalAllocNoChecks_FastPath now leaves the fast path for COM objects, large objects, GC stress, allocation tracking and allocation logging. That looks like a real fix, but it affects every caller of that path, not just func-eval. It's worth calling out in the description or moving to its own PR.
• The native/managed interface could be much smaller. FunctionEvaluation currently uses nine QCalls. Only GetMethod, PrepareResult and GetArgumentType need to load types, run class constructors, allocate or throw, and they could become a single prepare call.
  • GetArgument, ReadPrimitiveRegister and GetObject are plain reads of IPC data, the captured context, or arrays native already GC-protects (pObjects, pInteriors). Native could fill them into Context before calling InvokeFunction, as it already does for register-held struct snapshots, and Context.Objects already exists for the managed side.
  • WriteRegister and WriteObjectRegister both only call SetFuncEvalRegisterValue. Managed code could write into staging slots and mark them dirty, and native could apply them after the UCO returns. The no-exception-handler semantics wouldn't change: nothing is applied when the call throws.
    Speed isn't the point here. The point is a smaller interface and less per-argument back-and-forth between native and managed code. This could be a follow-up.
• Big-endian builds. RegisterFuncEvalSlot(&pArg->argHome.reg1Value, ...) uses GetNativeAddress on an 8-byte Portable<>, which static_asserts under DBG_BYTE_SWAP_REQUIRED (dbgportable.h). CoreCLR doesn't support big-endian targets, so this matters only for that configuration.
• Unused WASM thunks. The reverse thunks for Debugger.InvokeFunction in callhelpers-reverse.cpp are generated but can't be called on WASM, because func-eval is compiled out there. They do look consistent with the generator.
• VM and cDAC disagree on unexpected type kinds. For an ExternalMemoryHandle type that's a type descriptor other than byref, pointer or function pointer, the VM asserts and reports nothing. The cDAC reports it as an object slot. No current caller creates one.
• Testing. No in-repo tests cover func-eval, and no CI leg runs it. The description also says builds and tests were not repeated after the final rebase. Re-running the internal MDbg matrix on the final head would help, especially with cases for findings 1 and 2: repeatedly evaluating a struct-returning method while watching ExternalMemoryHandle count or memory, and evaluating a struct instance method from a caller frame.

What looked solid

• Managed side:
  • The emitted IL is consistent by slot across newobj and existing-instance constructors, receivers, by-ref copy-back, ref-struct locals and result storage.
  • The Context and NativeArgument layouts match native and are checked by DEFINE_FIELD_U.
  • Register value types larger than 8 bytes are rejected natively before managed code runs.
  • Handle arguments are always flagged as memory, so the receiver and copy-back checks are correct for them.
• GC safety: objects, interior pointers, homes and register struct snapshots are all rooted before the first GC-triggering step, and primitives are captured before class constructors can run.
• External memory: handles are reported per kind (byref slots as interior pointers, value types through ReportPointersFromValueType, pointers not at all), and each handle's loader allocator is reported during promotion.
• cDAC contract: changing it in place under the same version looks fine, since it was added after the 11.0 release branches and hasn't shipped.

CI

None of the 6 failing checks look related to this PR:

• The x86 CdacStressTest PrestubMethodFrame mismatch fails the same way on main.
• The x86 WindowsThreadPool.Tests access violation in PgoManager::getPgoInstrumentationResults and the SslStreamTlsResumeTests failure are known issues.
• The two ProcessWaitingTests failures have failure rates of 0.07% and 0.03%.
• The Mono Directed_r failure is a Helix dead-letter.

---

Note

This review was generated with Claude Opus 5.5. Findings 1–3 were checked against the code at 22e799b1c3e. Findings 4–6 and some of the smaller points come from AI analysis that wasn't independently checked. No code was built or run.

[jkoritzinsky]

@am11 addressed the feedback from your agent. It found some legit problems, much appreciated!

[jkotas]

Why is that? The debugger - that is work in progress - is going to need it.