feat: Add support for native runtimes

[robin-moss]

Making use of PluginImporter to allow us to configure the compatibility of native libraries to different platforms

[robin-moss]

We chose to use SQLite as it's small and easy to work with, but on linux it's already there so it picks up my system SQLite :(

Looking for alternative options.

We've also tried with gRPC, but it has a lot of additional magic for loading native files which means the NuGet packages are named incorrectly when running from Unity. Once we've got native runtimes working we can look at gRPC (which we need/want)

[robin-moss]

@jwittner took a bit of effort but looks like we've got it working. We had to add a playtest and some asmdefs 🙁 (details below).

To keep everything using internal we used the InternalsVisibleTo, but very open to discussing this more 🙂

Play Mode Test:

We tried to run the test within the NUnit tests but Unity just wouldn't find the newly imported libraries, however, it does work with a play mode test.

We had some arguments with the play mode tests and uninstalling, hence the LoadConfig part not really sure whats happening here.

[robin-moss]

Updating on testing this:

• Linux - works
• Mac OS - Works
• Windows - Fails :(

Some more work to set the CPU on the editor DLL, but even with this Unity complains (once) about two identical DLLs, after that one complaint it then works 🤔 so we need to figure out how to stop Unity for trying to import the two DLLs into the editor until we're ready, although that might not be possible as we are directly importing the assets.

[Twometer]

In a project I'm working on it would be really useful to have this working, so I tried finding the issue:

I've noticed that for me it repeatedly doesn't work ("Compiler errors must be fixed before you can enter play mode") with your version, even with setting the compatibility, because the Editor settings still are at Any CPU. I've added the following to the importer (Windows Only):

                    if (compatibleTargets.Contains(BuildTarget.StandaloneWindows))
                    {
                        plugin.SetEditorData("CPU", "x86");
                    }
                    else if (compatibleTargets.Contains(BuildTarget.StandaloneWindows64))
                    {
                        plugin.SetEditorData("CPU", "x86_64");
                    }
                    

as well as doing a

AssetDatabase.Refresh();

It still writes one or two "found the library twice" errors to the console, but if I ignore them and just run a simple SQLite playtest:

            var version = Marshal.PtrToStringAnsi(sqlite3_libversion());
            Assert.That(version, Is.EqualTo(_expectedVersion));

everything seems to work.

My full code is here, but it's kind of messy lol

Maybe this helps?

[robin-moss]

In a project I'm working on it would be really useful to have this working, so I tried finding the issue:

I've noticed that for me it repeatedly doesn't work ("Compiler errors must be fixed before you can enter play mode") with your version, even with setting the compatibility, because the Editor settings still are at Any CPU. I've added the following to the importer (Windows Only):

                    if (compatibleTargets.Contains(BuildTarget.StandaloneWindows))
                    {
                        plugin.SetEditorData("CPU", "x86");
                    }
                    else if (compatibleTargets.Contains(BuildTarget.StandaloneWindows64))
                    {
                        plugin.SetEditorData("CPU", "x86_64");
                    }
                    

as well as doing a

AssetDatabase.Refresh();

It still writes one or two "found the library twice" errors to the console, but if I ignore them and just run a simple SQLite playtest:

            var version = Marshal.PtrToStringAnsi(sqlite3_libversion());
            Assert.That(version, Is.EqualTo(_expectedVersion));

everything seems to work.

My full code is here, but it's kind of messy lol

Maybe this helps?

Thanks @Twometer I will give that a shot, the Windows side is taking longer because I don't actually have access to a Windows machine most of the time.

With the current state we saw that using NuiGet to install causes the error, but it then ran fine so we've been trying to fix the test.

🤦 I just checked I have a local change that does exactly what you've suggested, but because we hadn't got the tests working I hadn't pushed it (will push in a minute).

[robin-moss]

Tests are now working on Windows, we needed to ignore the error message to allow the tests to run.

@jwittner what are the next steps to getting this PR merged?

[Twometer]

@robin-moss Could you try if it works with EmguCV?
I've tried that (but on Unity 2020.3), and got a DllNotFoundException for the cvextern.dll when using functions from Emgu, but I don't know if the problem is with my broken branch, or if there is another issue with the importer.

[robin-moss]

@robin-moss Could you try if it works with EmguCV? I've tried that (but on Unity 2020.3), and got a DllNotFoundException for the cvextern.dll when using functions from Emgu, but I don't know if the problem is with my broken branch, or if there is another issue with the importer.

I can't easily test as I don't have a windows machine to test with, however, note that EmguCV also requires a runtime package to be installed as well please make sure you've done that. If you are still having a problem using it I suggest trying to call the DLL directly from C# to see if the new C# code can find the DLL, if it can then there is some magic in the package looking for a DLL with a different name (it's what gRPC does)

[Twometer]

I had installed the Runtime Packages for EmguCV, but looking at the source code it seems as if they do a lot of library loading magic...

[jwittner]

@robin-moss sorry I've been slammed the last couple of weeks. Next week I'll review the code, pull down the branch, do some tests and we'll go from there if that's ok. Thanks again for what's looking like an awesome contribution for a long awaited capability!

[robin-moss]

Sorry to chase @jwittner, we're using a locally compiled version for now but would like to switch to the official release

[jwittner]

Sorry for the delays, work and holidays took my attention away. Pulling your branch in now and I'll do some testing. =)

[robin-moss]

We've been using this change locally for about three months and it's been working great, there is the known "issue" with Windows where you cannot unload C++ binaries from Unity once loaded (and cannot delete them) but I don't think it's possible to resolve this issue as it's literally a limitation of Windows

[Twometer]

It is possible to unload C++ binaries from a Windows application with the FreeLibrary call, but I don't know if/how that works in the context of Unity.

[robin-moss]

To use that we would need to make setup OS-specific native DllImports so we could unload the native libraries (assuming this doesn't break Unity Editor).

This is only really an issue because Windows doesn't let us delete files that are in use :(

[robin-moss]

The latest push is a major change from how we were modifying the metafiles, I noticed that during Unity startup the calls to import an asset were failing so switched to modifying assets after Unity has imported them. This seems much more reliable than the trying to force synchronous imports

https://docs.unity3d.com/ScriptReference/AssetPostprocessor.OnPostprocessAllAssets.html

[robrab2000]

Hello :) I was curious if there was likely to be any movement on this getting merged?

[JoC0de]

@robrab2000 It is the biggest think left "on the list" but it is not trivial. We can add the support as it is implemented in this branch. But it will never work for "every" package. Packages that have native binaries often also have custom import steps (MSBuild Tasks) that we cant run. So we can't ensure that the NuGet package will work inside Unity. E.g. the nuget package need to contain one native binary for each target platform if one is missing it wouldn't work on the device. So it is generally not a good think to import packages with nativ dependencies. I am not sure how we can make this clear so users don't get into the "It works on my Machine" issue.

[robrab2000]

If there is an implementation which works for some packages and not for others, could it not be like an 'experimental' flag in the settings? Regarding platforms, perhaps you could just indicate beforehand what platform you would like. in fact perhaps there could be a separate runtime extraction tool where you can extract a runtime for your platform (one at a time so that there is no conflict between them before you get a chance to select a platform for it in the inspector).

Maybe I'm being silly and there are good reasons that these approaches wouldn't work?

[robin-moss]

We've been using a fork of this repo with the native stuff in for quite awhile.. mostly for gRPC support though.

A better way to do this is to use an asset import processor that can prevent the import before unity even tries to load it. As well as doing things like disabling auto importing be default.

But that's also a major departure from how this tool works (or worked).

My eventual plan was to replace the tool completely with just an asset processor and real nuget cli but just haven't had the time.

[JoC0de]

@robrab2000 Yes more or less we can implement it and show a warning so the User is aware of the fact that it contains native dependencies, and listing all platforms that are included inside the NuGet package so he is aware of possible missing support of target platforms.

[chubei-urus]

Hi @robin-moss may I ask what's the fork that you've been using? I checked https://github.com/robin-moss/NuGetForUnity but it's quite some commits behind so I'm not sure if it's the correct one.

[robin-moss]

That is the fork we're using, I hadn't updated against this repo in quite some time and looks like it's gone through some serious changes making it much harder to rebase my changes :(

I don't think I'll rebase this change on the latest version even if I want V3 Nuget support, just a lot of effort for something that is currently working ok for us