Marshal should be able to handle generic types

[masonwheeler]

I bet this was a big hassle back when generics were introduced, but now, it seems a bit ridiculous that Marshal can't handle generic types.

It's been considered good practice for years now to use Func<> and Action<> declarations rather than declaring named custom delegate types, but Marshal.GetDelegateForFunctionPointer doesn't allow you to follow this practice. And it should be simple to create a generic typed IntPtr-to-struct for working with native functions that take or return struct pointers, but the Marshal won't allow that either.

Is there any technical reason why Marshal can't handle generic types? Or was it just yet another case of "too much work to do in the limited time we had to get C# 2 out the door"? Because if it's the latter, ISTM it's about time we rectify that.

[jakesays-old]

comment.

[yizhang82]

@masonwheeler Thanks for bringing this up. At that point not supporting generics is mostly a decision to scope the work, until WinRT came along and we had to support generic interfaces and delegates in WinRT. I tend to agree that the limitations seems rather artificial (and in some cases, inconsistent), and as you've pointed out not supporting Func/Action case seems particularly inconvenient (I must admit I've run into them myself). The main work of implementing this in the runtime is hook up / manage the marshalling data structures to the generic MethodTable instances correctly, for all the scenarios that we support. I'm not an expert on type loader so I can't comment on how much work that is. I would think that a better time to implement this is when we have merged all the interop implementations in multiple runtimes (desktop CLR, CoreCLR, .NET Native) into one. It would've been much easier to implement this in code generators (MCG) and we only need to do it once :)

[masonwheeler]

@yizhang82 What is CoreCLR using right now? I sort of have the vague impression that it's using MCG but not everything is in place yet, but I could be wrong there...

[yizhang82]

@masonwheeler CoreCLR's interop implementation is entirely in the runtime itself - COM interop (RCW/CCW), pinvoke, data marshalling, etc. The most interesting files are in runtimecallablewrapper.cpp (RCW), comcallablewrapper.cpp (CCW), mlinfo.cpp (signature parsing), ilmarshaler.cpp (marshaling code), dllimport.cpp (interop stubs), dllimportcallback.cpp (reverse pinvoke).

[xoofx]

Hey, bumping this old issue... running into a situation where we have quite some code in Unity that is actually passing delegate with generics to native and calling it back. As we can directly use Mono API for that, we are able to call this callback even if they have generics...
But in case we would like to use .NET CoreCLR, we would not be able to do this easily in a standard way using regular pinvoke.

So the question is: How much work is it to support generics for the various p/invoke scenarios?

cc: @jkotas

[masonwheeler]

@xoofx So Mono supports it, or it doesn't exactly but you're haxxoring up a workaround?

[xoofx]

@xoofx So Mono supports it, or it doesn't exactly but you're haxxoring up a workaround?

Yeah, mono p/invoke does not support it (actually I never tried, but I don't think), but as we are using the mono VM api from C++, we can call any C# code we want.

[xoofx]

Think of it as a manual reverse pinvoke where we are writing in C++ what would have to do an auto-generated reverse pinvoke callback

[masonwheeler]

Gah! That is something probably best not thought about! 😛

[xoofx]

Gah! That is something probably best not thought about! 😛

Yeah, that's why. I would like that we move out of this kind of hacks, but in order to do that, we should have a support from the regular VM. One way to solve this on our side would be to have a Roslyn compiler plugin to rewrite the code internally to non generic version... but I would prefer a standard support from the runtime for this rather than having an ugly workaround

[jkotas]

So the question is: How much work is it to support generics for the various p/invoke scenarios?

For blittable types, it is "just work" to support it for existing interop (not a small amount, but doable in principle). For non-blittable types, there is a lot of ambiguity in how it should even work.

@xoofx Does your case have blittable types only, or does it have arbitrary types?

[xoofx]

@xoofx Does your case have blittable types only, or does it have arbitrary types?

Blittable would be fine (we may not have fully this setup today, but we would have anyway)

[masonwheeler]

My original request was about generic delegate types, but I did have in mind using only blittable types as the arguments to the generic functions.

[jkotas]

BTW: For callbacks, a better solution may be NativeCallbableAttribute - dotnet/coreclr#1566 . Enabling that for generic blitable types should be much easier. NativeCallbableAttribute is not really usable from C# directly today, but that's going to change with https://github.com/dotnet/csharplang/blob/master/proposals/intrinsics.md

[tannergooding]

Given that C# has now merged the unmanaged constructed types feature, would it be worth looking at enabling the same types of generic structs to be used in interop scenarios (this basically limits it to "blittable" generic structs)?