.NET: Add support for producing a self-contained native framework/library

[rolfbjarne]

This is a meta/placeholder issue for tracking the interest in and the work required to produce a self-contained native framework or library in .NET.

In particular we're not committing to actually implementing this, no decision has been made yet.

There are several questions that would have to be answered and solutions designed before we can even think about starting to work on this.

Open questions:

• Exactly what is exposed from managed code? Exposing only methods with the [UnmanagedCallersOnly] attribute would be the simplest, but it would also not enable some more complex scenarios (such as subclassing a managed class in Objective-C for instance).
• What happens if an app wants to use two shared libraries, built with different versions of the SDK/Runtime/etc?
• ...

Random thoughts:

• It's probably best to create a framework (and not a dylib), because dylibs only work on macOS, while frameworks work everywhere.
• We'd have to make sure it works with both MonoVM and CoreCLR.

Ref:

• Support AOT that target shared libraries on Mono for the .NET 7 time frame dotnet/runtime#68775
• https://github.com/mono/Embeddinator-4000

[hugener]

I work for a larger organization that is reviewing how to build our next software platform.

Our current platform was built with .NET. Not being able to produce native libraries for mobile platforms (Android, iOS), such that our customers can integrate them into their own native mobile apps out of the same .NET code base, may be a deal breaker for .NET. Without this capability, we will face the decision whether to stay with .NET or to move to another platform like Rust. We would like to therefore state that we are highly interested in this capability in .NET!

[dasmihir20]

My organisation also has shared code base which is getting used from desktop app and mobile app. For Mobile app we were using https://github.com/mono/Embeddinator-4000. However for MAUI(dotnet 6.0 onwards) library, we are clueless.[UnmanagedCallersOnly] would be enough for our case.

[hugener]

Hi @rolfbjarne
Is there any update on this topic?
I see the issue here: dotnet/runtime#68775 is moved to the 8.0.0 milestone, which would be fine, but it would be good to know what the plans are.

[rolfbjarne]

@hugener right now this is not on our roadmap to be implemented for .NET 8 (which may change, but as of right now that's what I know).

[ceztko]

I'm in a very similar situation to @dasmihir20: my company would seriously evaluate Native AOT technologies to produce libraries, and would be especially interested in iOS/Android, because .NET runtime is not easy/impossible to deploy in these platforms.

It's probably best to create a framework (and not a dylib), because dylibs only work on macOS, while frameworks work everywhere.

Definitely. Consider dylibs can be easily converted to framework by patching the file as a very minimum, plus one would have to define a manifest Info.plist and add a C header. I would love to understand if I can experiment with such technologies today somehow: is it already possible to generate a bare iOS ARM dylib with the API defined by the [UnmanagedCallersOnly] attributes? If so, how? For the reference I just asked a similar question in dotnet/runtime#68775 .

[rolfbjarne]

is it already possible to generate a bare iOS ARM dylib with the API defined by the [UnmanagedCallersOnly] attributes?

It might be possible, but it's not going to be easy.

The process would involve building a complete app bundle, and then looking that the intermediate native (.m/.s) files and manually recompiling them yourself into a dylib/framework.

You'd probably run into a lot of problems along the way, and I'm not sure they'll all be solvable.

[ceztko]

The process would involve building a complete app bundle, and then looking that the intermediate native (.m/.s) files and manually recompiling them yourself into a dylib/framework.

Maybe what's being cooking on Native AOT involves a completely different technology? Looking at what Native AOT does when PublishAot is enabled points to Microsoft.NETCore.Native.targets which calls this ILCompiler which really translates the IL bytecode to native code, apparently using the very same RyuJIT JIT compiler that is at the base of .NET desktop runtimes. The native object produced is then linked in a shared object using the platform linker. Looking at the amount of stuff already done, I don't understand why you wonder about Objective-C extensibility: the very first target should be have a C ABI shared library, using that framework. I would not surprise too much if all the basic blocks to produce a valid ios dylib are there: there's unix/darwin and ARM64 support. If what was is missing is some tooling for the linking part, then the 8.0.0 milestone doesn't look like a dream. Maybe there's a conflict of responsibility of who is charge of doing this for iOS?

[rolfbjarne]

The process would involve building a complete app bundle, and then looking that the intermediate native (.m/.s) files and manually recompiling them yourself into a dylib/framework.

Maybe what's being cooking on Native AOT involves a completely different technology? Looking at what Native AOT does when PublishAot is enabled points to Microsoft.NETCore.Native.targets which calls this ILCompiler which really translates the IL bytecode to native code, apparently using the very same RyuJIT JIT compiler that is at the base of .NET desktop runtimes. The native object produced is then linked in a shared object using the platform linker. Looking at the amount of stuff already done, I don't understand why you wonder about Objective-C extensibility: the very first target should be have a C ABI shared library, using that framework. I would not surprise too much if all the basic blocks to produce a valid ios dylib are there: there's unix/darwin and ARM64 support. If what was is missing is some tooling for the linking part, then the 8.0.0 milestone doesn't look like a dream. Maybe there's a conflict of responsibility of who is charge of doing this for iOS?

Yes, NativeAOT is a very different process, but at the moment NativeAOT doesn't support iOS, so that's not an option if you want to hack something together today.

[huzaifacontour]

Any update on this? I'm facing apple store issue for uploading the dylibs, and I don't know why my framework is not working when I converted dylibs libs into it.

I'm using GDAL libraries, I upgraded them and now they just came up with dynamic kind with C# bindings.

[rolfbjarne]

@huzaifacontour it might be possible to publish to a native library using NativeAOT: https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/?tabs=net8plus%2Cwindows#build-native-libraries, but you'd only be able to use the .NET base class library (and your own custom code), but none of the iOS-specific bindings we ship in the iOS SDK.

[huzaifacontour]

@rolfbjarne I resolved the above issue by converting the libraries to framework of two archs x86_64 and arm64, It's working with this but when I converted the framework to xcFramework and added as a native reference I'm having access to path denied:
bin.log

[rolfbjarne]

@rolfbjarne I resolved the above issue by converting the libraries to framework of two archs x86_64 and arm64, It's working with this but when I converted the framework to xcFramework and added as a native reference I'm having access to path denied: bin.log

Please file a new issue to avoid making things confusing here, and also get a binary build log (https://github.com/xamarin/xamarin-macios/wiki/Diagnosis#binary-build-logs), that will make diagnosis much easier.