Handle returned GetMainProgramHandle() not sufficient for resolving statically linked native symbols on iOS

[stanger]

Description

Based on the discussion in #57610 we were under the impression that using GetMainProgramHandle() in tandem with NativeLibrary.SetDllImportResolver() would be a replacement for having to modify third party packages to use [DllImport("_Internal")] on iOS. So far in testing on .NET 7 this has not been the case.

I created a sample native iOS library (libsayhello) and a wrapping C# lib which handles [DllImport("libsayhello')]. Then within a sample Maui app I added Interop code to allow my lib to be statically linked into the main application at compile time, then attempted to catch the resolution of 'libsayhello' in the import resolver and return the handle returned by GetMainProgramHandle().

Reproduction Steps

A test app showing the behavior is located at stanger/ResolverTest.

Expected behavior

DllImport calls for 'libsayhello' should be able to resolved to their statically linked symbols in the main binary.

Actual behavior

DllNotFoundException

Regression?

No response

Known Workarounds

No response

Configuration

.NET: 7.0.100-rc.2.22477.23
iOS 16 and MacOS 13 (originally discovered on iOS)
Tested on x64 and ARM64

Other information

No response

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Based on the discussion in #57610 we were under the impression that using GetMainProgramHandle() in tandem with NativeLibrary.SetDllImportResolver() would be a replacement for having to modify third party packages to use [DllImport("_Internal")] on iOS. So far in testing on .NET 7 this has not been the case.

I created a sample native iOS library (libsayhello) and a wrapping C# lib which handles [DllImport("libsayhello')]. Then within a sample Maui app I added Interop code to allow my lib to be statically linked into the main application at compile time, then attempted to catch the resolution of 'libsayhello' in the import resolver and return the handle returned by GetMainProgramHandle().

Reproduction Steps

A test app showing the behavior is located at stanger/ResolverTest.

Expected behavior

DllImport calls for 'libsayhello' should be able to resolved to their statically linked symbols in the main binary.

Actual behavior

DllNotFoundException

Regression?

No response

Known Workarounds

No response

Configuration

.NET: 7.0.100-rc.2.22477.23
iOS 16 and MacOS 13 (originally discovered on iOS)
Tested on x64 and ARM64

Other information

No response

Author:	stanger
Assignees:	-
Labels:	area-System.Runtime.InteropServices, untriaged
Milestone:	-

[jkoritzinsky]

I see the problem. We definitely should have caught this earlier as this was the end-to-end scenario...

Sometimes, the value returned by GetMainProgramHandle() is IntPtr.Zero, as that's the handle to use for this case. However, SetDllImportResolver expects the delegate to return IntPtr.Zero for the failure case. As a result, using GetMainProgramHandle with SetDllImportResolver as it exists today has a fundamental problem on platforms where IntPtr.Zero is the correct handle for this case.

I see a few solutions here:

1. Introduce an API specifically for resolving DllImports to the main program handle (what we were trying to avoid).
2. Introduce an overload of SetDllImportResolver that takes a delegate that returns IntPtr? instead of IntPtr. The user experience here would be pretty rough though as overload resolution would prefer the IntPtr overload for target-typed lambdas unless we got something like [API Proposal]: Add ObsoleteAttribute.BinaryCompatOnly #65965 (I don't think ref-assembly tricks would work here well).

[am11]

Sometimes, the value returned by GetMainProgramHandle() is IntPtr.Zero

Going by:

runtime/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/NativeLibrary.cs

Line 261 in 02c3e95

if (result == IntPtr.Zero)

it throws an exception instead of returning IntPtr.Zero.

[jkoritzinsky]

In which case, I'm not sure what would be causing a DllNotFoundException specifically.

@stranger, can you confirm what GetMainProgramHandle is returning?

[stanger]

This is the value I see in the debugger is:

handle 0xfffffffffffffffe

[jkoritzinsky]

Hmm. In that case, I'm not sure the issue off the top of my head.

Can you get the stack trace for where the exception is thrown from? I don't have a Mac handy at the moment.

Also, can you try adding __attribute__((visibility(default))) to the native definition of your say_hello method?

[stanger]

The stack trace isn't overly verbose and the inner exception was null.

System.DllNotFoundException: libsayhello at ResolverSample.MainPage.OnCounterClicked(Object sender, EventArgs e) in /Users/stanger/Workspaces/Enclave/ResolverSample/src/ResolverSample/MainPage.xaml.cs:line 17

I have also updated the sample lib definition with __attribute__((visibility("default")))