[Java.Base-Tests] Test Java-to-Managed invocations for Java.Base (#975)

Commit bc5bcf4 (Desktop `Java.Base` binding) had a TODO:

> ~~ TODO: Marshal Methods ~~
>
> Marshal methods are currently skipped.  Java-to-managed invocations
> are not currently supported.

No marshal methods means no facility for Java-to-managed invocations.

Add a new `tests/Java.Base-Tests` unit test assembly, and test
Java-to-managed invocations.

This requires touching and updating ~everything. 😅

Update `Java.Interop.dll` to contain a new
`Java.Interop.JniMethodSignatureAttribute` custom attribute.

Update `generator` to begin emitting `[JniMethodSignature]` on bound
methods.  This is necessary so that `jcw-gen` knows the JNI method
signature of Java methods to emit.  As part of this, *remove* the
`JniTypeSignatureAttr` type added in bc5bcf4; trying to follow the
`JniTypeSignatureAttr` pattern for `JniMethodSignatureAttr` would
make for more intrusive code changes.  Instead, "re-use" the existing
`RegisterAttr` type, adding a `RegisterAttr.MemberType` property so
that we can select between Android `[Register]` output vs. "Desktop"
`[JniTypeSignature]` and `[JniMethodSignature]` output.  This in turn
requires updating many `generator-Tests` artifacts.

Update `Java.Interop.Tools.JavaCallableWrappers` to look for
`Java.Interop.JniTypeSignatureAttribute` and
`Java.Interop.JniMethodSignatureAttribute`.  This allows
`jcw-gen Java.Base-Tests.dll` to generate Java Callable Wrappers for
`Java.BaseTests.MyRunnable`.

Update `Java.Interop.Export.dll` so that `MarshalMemberBuilder`
doesn't use `Expression.GetActionType()` or
`Expression.GetFuncType()`, as .NET doesn't support the use of
generic types in [`Marshal.GetFunctionPointerForDelegate()`][0].
Instead, we need to use `System.Reflection.Emit` to define our own
custom delegate types.

~~ Comparison with Android ~~

Android bindings (both Classic Xamarin.Android and
.NET SDK for Android) use `generator`-emitted "connector methods"
which are specified in the `[Register]` attribute (see also 99897b2):

	namespace Java.Lang {
	    [Register ("java/lang/Object" DoNotGenerateAcw=true)]
	    partial class Object {
	        [Register ("equals", "(Ljava/lang/Object;)Z", "GetEquals_Ljava_lang_Object_Handler")]
	        public virtual unsafe bool Equals (Java.Lang.Object? obj)
	            => …

	        static Delegate GetEquals_Ljava_lang_Object_Handler()
	            => JNINativeWrapper.CreateDelegate((_JniMarshal_PPL_Z) n_Equals_Ljava_lang_Object_);
	        static bool n_Equals_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this, IntPtr native_obj)
	            => …
	    }
	}

`jcw-gen` emits the "connector method" into the resulting
Java Callable Wrappers for appropriate subclasses:

	String __md_methods =
	    "n_equals:(Ljava/lang/Object;)Z:GetEquals_Ljava_lang_Object_Handler\n";

At runtime, `AndroidTypeManager.RegisterNativeMembers()` will lookup
`Object.GetEquals_Ljava_lang_Object_Handler()` and invoke it.
The returned `Delegate` instance is provided to
`JNIEnv::RegisterNatives()`.

The problem with this approach is that it's inflexible: there is no
way to participate in the "connector method" infrastructure to alter
marshaling behavior.  If you need custom behavior, e.g.
`Android.Graphics.Color` customizations, you need to update
`generator` and rebuild the binding assemblies.

(This has been "fine" for the past 10+ years, so this hasn't been a
deal breaker.)

For `Java.Base`, @jonpryor wants to support the custom marshaling
infrastructure introduced in 77a6bf8.  This would allow types to
participate in JNI marshal method ("connector method") generation
*at runtime*, allowing specialization based on the current set of
types and assemblies.

This means we *don't* need to specify a "connector method" in
`[JniMethodSignatureAttribute]`, nor generate them.

	namespace Java.Lang {
	    [JniTypeSignatureAttribute("java/lang/Object", GenerateJavaPeer=false)]
	    partial class Object {
	        [JniMethodSignatureAttribute("equals", "(Ljava/lang/Object;)Z")]
	        public virtual unsafe bool Equals (Java.Lang.Object? obj)
	            => …

	        // No GetEquals_Ljava_lang_Object_Handler()
	        // No n_Equals_Ljava_lang_Object_()
	    }
	}

This should result in smaller binding assemblies.

The downside is that runtime costs *increase*, significantly.
Instead of looking up and invoking a "connector method", the method
must be *generated* via System.Linq.Expressions expression trees,
then compiled into IL, then JIT'd, all before it can be used.

We have no timing data to indicate how "bad" this overhead will be.

As always, the hope is that `tools/jnimarshalmethod-gen` (176240d)
can be used to get the "best of both worlds": the flexibility of
custom marshalers, without the runtime overhead.  But…

  * #14
  * #616

At this point in time, `jnimarshalmethod-gen` *cannot* work under
.NET 6.  This will need to be addressed in order for custom marshalers
to be a useful solution.

`Java.Base` will use "connector method"-less bindings to act as a
"forcing function" in getting `jnimarshalmethod-gen` working in .NET.

[0]: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal.getfunctionpointerfordelegate?view=net-6.0

Author: jonpryor

Java.Interop.sln

@@ -105,6 +105,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Interop.Tools.JavaType
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Base", "src\Java.Base\Java.Base.csproj", "{30DCECA5-16FD-4FD0-883C-E5E83B11565D}"
 EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Base-Tests", "tests\Java.Base-Tests\Java.Base-Tests.csproj", "{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}"
+EndProject
 Global
 	GlobalSection(SharedMSBuildProjectFiles) = preSolution
 		src\Java.Interop.NamingCustomAttributes\Java.Interop.NamingCustomAttributes.projitems*{58b564a1-570d-4da2-b02d-25bddb1a9f4f}*SharedItemsImports = 5
@@ -296,6 +298,10 @@ Global
 		{30DCECA5-16FD-4FD0-883C-E5E83B11565D}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{30DCECA5-16FD-4FD0-883C-E5E83B11565D}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{30DCECA5-16FD-4FD0-883C-E5E83B11565D}.Release|Any CPU.Build.0 = Release|Any CPU
+		{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE
@@ -346,6 +352,7 @@ Global
 		{B173F53B-986C-4E0D-881C-063BBB116E1D} = {0998E45F-8BCE-4791-A944-962CD54E2D80}
 		{11942DE9-AEC2-4B95-87AB-CA707C37643D} = {271C9F30-F679-4793-942B-0D9527CB3E2F}
 		{30DCECA5-16FD-4FD0-883C-E5E83B11565D} = {0998E45F-8BCE-4791-A944-962CD54E2D80}
+		{CB05E11B-B96F-4179-A4E9-5D6BDE29A8FC} = {271C9F30-F679-4793-942B-0D9527CB3E2F}
 	EndGlobalSection
 	GlobalSection(ExtensibilityGlobals) = postSolution
 		SolutionGuid = {29204E0C-382A-49A0-A814-AD7FBF9774A5}

Makefile

@@ -32,6 +32,9 @@ TESTS = \
 	bin/Test$(CONFIGURATION)/Java.Interop.Tools.Generator-Tests.dll \
 	bin/Test$(CONFIGURATION)/Xamarin.SourceWriter-Tests.dll
 
+NET_TESTS = \
+	bin/Test$(CONFIGURATION)-net6.0/Java.Base-Tests.dll
+
 PTESTS = \
 	bin/Test$(CONFIGURATION)/Java.Interop-PerformanceTests.dll
 
@@ -44,6 +47,7 @@ run-all-tests:
 	r=0; \
 	$(MAKE) run-tests                 || r=1 ; \
 	$(MAKE) run-test-jnimarshal       || r=1 ; \
+	$(MAKE) run-net-tests             || r=1 ; \
 	$(MAKE) run-ptests                || r=1 ; \
 	$(MAKE) run-java-source-utils-tests     || r=1 ; \
 	exit $$r;
@@ -123,6 +127,11 @@ run-tests: $(TESTS) bin/Test$(CONFIGURATION)/$(JAVA_INTEROP_LIB)
 	$(foreach t,$(TESTS), $(call RUN_TEST,$(t),1)) \
 	exit $$r;
 
+run-net-tests: $(NET_TESTS) bin/Test$(CONFIGURATION)-net6.0/$(JAVA_INTEROP_LIB)
+	r=0; \
+	$(foreach t,$(NET_TESTS), dotnet test $(t) || r=1) \
+	exit $$r;
+
 run-ptests: $(PTESTS) bin/Test$(CONFIGURATION)/$(JAVA_INTEROP_LIB)
 	r=0; \
 	$(foreach t,$(PTESTS), $(call RUN_TEST,$(t))) \

build-tools/automation/templates/core-tests.yaml

@@ -130,6 +130,15 @@ steps:
     arguments: bin/Test$(Build.Configuration)$(NetCoreTargetFrameworkPathSuffix)/Java.Interop-PerformanceTests.dll
   continueOnError: true
 
+- task: DotNetCoreCLI@2
+  displayName: 'Tests: Java.Base'
+  condition: or(eq('${{ parameters.runNativeDotnetTests }}', 'true'), eq('${{ parameters.runNativeTests }}', 'true'))
+  inputs:
+    command: test
+    testRunTitle: Java.Base (net6.0 - ${{ parameters.platformName }})
+    arguments: bin/Test$(Build.Configuration)$(NetCoreTargetFrameworkPathSuffix)/Java.Base-Tests.dll
+  continueOnError: true
+
 - task: DotNetCoreCLI@2
   displayName: 'Tests: java-source-utils'
   inputs:

src/Java.Base-ref.cs

@@ -4,6 +4,7 @@
 using System.Linq;
 using System.Linq.Expressions;
 using System.Reflection;
+using System.Reflection.Emit;
 using System.Text;
 
 using Java.Interop.Expressions;
@@ -241,15 +242,6 @@ public LambdaExpression CreateMarshalToManagedExpression (MethodInfo method, Jav
 
 		static Type GetMarshalerType (Type returnType, List<Type> funcTypeParams, Type declaringType)
 		{
-			// `mscorlib.dll` & `System.Core.dll` only provide Action<…>/Func<…> types for up to 16 parameters
-			if (funcTypeParams.Count <= 16) {
-				if (returnType != null)
-					funcTypeParams.Add (returnType);
-				return returnType == null
-					? Expression.GetActionType (funcTypeParams.ToArray ())
-					: Expression.GetFuncType (funcTypeParams.ToArray ());
-			}
-
 			// Too many parameters; does a `_JniMarshal_*` type exist in the type's declaring assembly?
 			funcTypeParams.RemoveRange (0, 2);
 			var marshalDelegateName = new StringBuilder ();
@@ -265,11 +257,58 @@ static Type GetMarshalerType (Type returnType, List<Type> funcTypeParams, Type d
 			}
 
 			Type marshalDelegateType = declaringType.Assembly.GetType (marshalDelegateName.ToString (), throwOnError: false);
+			if (marshalDelegateType != null) {
+				return marshalDelegateType;
+			}
 
+#if !NET
 			// Punt?; System.Linq.Expressions will automagically produce the needed delegate type.
 			// Unfortunately, this won't work with jnimarshalmethod-gen.exe.
 			return marshalDelegateType;
+#else   // NET
+			return CreateMarshalDelegateType (marshalDelegateName.ToString (), returnType, funcTypeParams);
+#endif  // NET
+		}
+
+#if NET
+		static object           ab_lock         = new object ();
+		static AssemblyBuilder  assemblyBuilder;
+		static ModuleBuilder    moduleBuilder;
+		static Type[]           DelegateCtorSignature;
+
+		static Type CreateMarshalDelegateType (string name, Type returnType, List<Type> funcTypeParams)
+		{
+			lock (ab_lock) {
+				if (assemblyBuilder == null) {
+					var aname       = new AssemblyName ("jni-marshal-method-delegates");
+					assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly (aname, AssemblyBuilderAccess.Run);
+					moduleBuilder   = assemblyBuilder.DefineDynamicModule (aname.Name!);
+
+					DelegateCtorSignature = new Type[] {
+						typeof (object),
+						typeof (IntPtr)
+					};
+				}
+				funcTypeParams.Insert (0, typeof (IntPtr));
+				funcTypeParams.Insert (0, typeof (IntPtr));
+				var typeBuilder = moduleBuilder.DefineType (
+					name,
+					TypeAttributes.Class | TypeAttributes.Public | TypeAttributes.Sealed | TypeAttributes.AnsiClass | TypeAttributes.AutoClass,
+					typeof (MulticastDelegate)
+				);
+
+				const MethodAttributes      CtorAttributes      = MethodAttributes.RTSpecialName | MethodAttributes.HideBySig | MethodAttributes.Public;
+				const MethodImplAttributes  ImplAttributes      = MethodImplAttributes.Runtime | MethodImplAttributes.Managed;
+				const MethodAttributes      InvokeAttributes    = MethodAttributes.Public | MethodAttributes.HideBySig | MethodAttributes.NewSlot | MethodAttributes.Virtual;
+
+				typeBuilder.DefineConstructor (CtorAttributes, CallingConventions.Standard, DelegateCtorSignature)
+					.SetImplementationFlags (ImplAttributes);
+				typeBuilder.DefineMethod ("Invoke", InvokeAttributes, returnType, funcTypeParams.ToArray ())
+					.SetImplementationFlags (ImplAttributes);
+				return typeBuilder.CreateTypeInfo ();
+			}
 		}
+#endif  // NET
 
 		static char GetJniMarshalDelegateParameterIdentifier (Type type)
 		{