[Java.Interop] Java-initiated instance activation

Fixes: #11

(Last seen nearly *two years ago* in commit 8c83f64...)

An IJavaPeerable implementation has two halfs: the managed peer and
the native peer (51f7862). The managed peer references the native
peer via IJavaPeerable.PeerReference, and an association from the
native peer to the managed peer is available through e.g.
JniRuntime.JniValueManager.PeekPeer(JniObjectReference).

Thus, the Very Important Question™: How is this association created?

From constructors.

Note the plural, and I don't mean constructor overloads.

*Both* managed peer and user peer constructors are (mostly [0])
responsible for setting up the assocation.

Managed peers do so via JniRuntime.JniValueManager.Construct()
(b3a2162; formerly SetObjectPeerReference() in 25de1f3):

	/*  1 */ public unsafe BindingConstructor ()
	/*  2 */    : base (ref *InvalidJniObjectReference, JniObjectReferenceOptions.None)
	/*  3 */ {
	/*  4 */    if (PeerReference.IsValid)
	/*  5 */        return;
	/*  6 */
	/*  7 */    var peer = JniPeerMembers.InstanceMethods.StartCreateInstance ("()V", GetType (), null);
	/*  8 */    JniEnvironment.Runtime.ValueManager.Construct (this, ref peer, JniObjectReferenceOptions.CopyAndDispose);
	/*  9 */    JniPeerMembers.InstanceMethods.FinishCreateInstance ("()V", this, null);
	/* 10 */ }

Line 7 creates the native peer, line 8 calls
JniRuntime.JniValueManager.Construct(), which calls
IJavaPeerable.SetPeerReference() with a GREF and creates the
association between `peer` and `this`, and line 9 finishes cleanup.

(What about lines 2, 4, and 5? Those are new; see below.)

Then we need "the other direction", when Java code performs the
construction. This process of calling a managed peer constructor with
a previously instantiated native peer instance is called *activation*.

In 8c83f64, activation support through the Java
ManagedPeer.runConstructor() API was stubbed and did nothing.

Implement ManagedPeer.runConstructor() so that when a Java User peer
is constructed, an instance of the associated managed peer is created.

This turns out to be rather complicated, for all manner of reasons.

First, the happy path:

	public MyJavaType (Args...)
	{
		com.xamarin.java_interop.ManagedPeer.runConstructor (
				MyJavaType.class,
				this,
				"AssemblyQualifiedNameOfManagedPeer",
				"`:`-separated AssemblyQualified signature of AssemblyQualifiedNameOfManagedPeer constructor",
				args...
	}

When Java code invokes `new MyJavaType(...)`,
ManagedPeer.runConstructor() is executed (8c83f64), which will call
ManagedPeer.RunConstructor().

ManagedPeer.RunConstructor() will find the specified managed peer
type, lookup the specified constructor based on the signature, marshal
the parameters, and invoke the specified constructor.

Pause briefly to see the problem...

The problem is that in this scenario, the Java instance is created
*first*. Consequently, the "normal" managed peer constructor
implementation of JniPeerMembers.InstanceMethods.StartCreateInstance()
WON'T WORK, as that creats a NEW Java instance, not an existing one.

Further complicating matters is that we want a "natural" experience,
not a special constructor signature, so how do we provide the
*existing* Java instance to the managed peer that will be created?

The answer? By violating all manner of "invariants" by using
Serialization-related APIs.

Let's rephrase things: Given a Java type J and a Managed Peer type M,
how do we associate an instance of J to an instance of M
*before constructing M*?

With FormatterServices.GetUninitializedObject(), which returns a blob
of zero-initialized GC memory of "type" M which hasn't been
constructed yet.

There be potential dragons here.

We then call IJavaPeerable.SetPeerReferenc() on this
*unconstructed* instance, then use Reflection to invoke the
constructor on the pre-allocated instance.

Thus:

	object m = FormatterServices.GetUninitializedObject(typeof(M));
	((IJavaPeerable) o).SetPeerReference (J_instance);
	constructorInfoOfM.Invoke(m, args);

(The stunning thing is that the above works at all!)

This allows the Java peer instance to be associated with a Managed
peer instance *before constructing the Managed peer*, which thus
allows invoking the desired Managed peer constructor.

Which brings us to the newly added lines 2, 4, and 5:

	/*  1 */ public unsafe BindingConstructor ()
	/*  2 */    : base (ref *InvalidJniObjectReference, JniObjectReferenceOptions.None)
	/*  3 */ {
	/*  4 */    if (PeerReference.IsValid)
	/*  5 */        return;

All "binding" constructors -- constructors which would normally use
JniPeerMembers.InstanceMethods.StartCreateInstance() and
JniRuntime.JniValueManager.Construct() -- must now call the
JavaObject(JniObjectReference&, JniObjectReferenceOptions) or
JavaException(JniObjectReference&, JniObjectReferenceOptions)
constructors (line 2), to check for a "previously" set PeerReference
value.  If there was a previously set PeerReference value -- i.e.
we're undergoing an activation-originated constructor call -- then the
constructor should STOP EXECUTING (lines 4-5).

Failure to do so will result in creating a new Java instance, and
associating the current managed peer instance to the *wrong* Java
instance. Don't Do That.

What about the less-than-happy path? That's when a virtual method is
invoked from a Java constructor and overridden in the managed peer.
This is described somewhat in 972c5bc and 3043d89.

The use of JNIEnv::AllocObject() (972c5bc) helps when the Managed
peer is constructed first. When the native peer is constructed first,
JNIEnv::AllocObject() can't help you.

In this scenario, the Managed peer instance is instead constructed
*multiple times*

  1.  Java: D.<init>()
  2.  Java: B.<init>()
  3.  Java: B.calledFromConstructor()
  4.  Java: D.n_calledFromConstructor();
  5.    C#: D.CalledFromConstructorHandler()
  6.    C#: JniRuntime.JniValueManager.CreateProxy() for (1)
            This invokes the
            D(JniObjectReference&, JniObjectReferenceOptions)
            constructor. If not present, throws.
  7.    C#: D.CalledFromConstructor()
  8.  Java: B.<init>() finishes
  9.  Java: D.<init>() calls ManagedPeer.runConstructor().
 10.    C#: ManagedPeer.RunConstructor() finds specified
            ConstructorInfo, invokes on instance created in (6).

Note the Managed peer constructor invocations in (6) and (10), both on
the same instance.

What's with this Reflection for
FormatterServices.GetUninitializedObject()?

FormatterServices.GetUninitializedObject() isn't present in any PCL
Profile (doh!), but it's the only mechanism I know of that allows me
to not require a special activation constructor.

Additionally, FormatterServices.GetUninitializedObject() IS supported
in CoreCL, so this should work on all interesting runtimes.

[0]:  One exception being "framework peers", which don't do anything
      special in their constructors to cause managed peers to be
      created.

Author: jonpryor

src/Java.Interop.Export/Java.Interop/ExportedMemberBuilder.cs

@@ -326,6 +326,33 @@ static Expression GetRuntime ()
 			var runtime = Expression.Property (null, env, "Runtime");
 			return runtime;
 		}
+
+		static  MethodInfo  FormatterServices_GetUninitializedObject    = Type.GetType ("System.Runtime.Serialization.FormatterServices", throwOnError: true)
+			.GetRuntimeMethod ("GetUninitializedObject", new[]{typeof (Type)});
+		static  MethodInfo  IJavaPeerable_SetPeerReference              = typeof (IJavaPeerable).GetRuntimeMethod ("SetPeerReference", new[]{typeof (JniObjectReference)});
+
+		public override Expression<Func<ConstructorInfo, JniObjectReference, object[], object>> CreateConstructActivationPeerExpression (ConstructorInfo constructor)
+		{
+			if (constructor == null)
+				throw new ArgumentNullException (nameof (constructor));
+
+			Func<object, object[], object>  mbi = constructor.Invoke;
+
+			var c   = Expression.Parameter (typeof (ConstructorInfo),       "constructor");
+			var r   = Expression.Parameter (typeof (JniObjectReference),    "reference");
+			var p   = Expression.Parameter (typeof (object[]),              "parameters");
+
+			var t   = Expression.Variable (typeof (Type),   "type");
+			var s   = Expression.Variable (typeof (object), "self");
+			var b   = Expression.Block (
+					new []{t, s},
+					Expression.Assign (t, Expression.Property (c, "DeclaringType")),
+					Expression.Assign (s, Expression.Call (FormatterServices_GetUninitializedObject, t)),
+					Expression.Call (Expression.Convert (s, typeof (IJavaPeerable)), IJavaPeerable_SetPeerReference, r),
+					Expression.Call (c, mbi.GetMethodInfo (), s, p),
+					s);
+			return Expression.Lambda<Func<ConstructorInfo, JniObjectReference, object[], object>> (b, new []{c, r, p});
+		}
 	}
 
 	static class JniSignature {

src/Java.Interop.Export/Tests/Java.Interop/ExportedMemberBuilderTest.cs

@@ -1,6 +1,7 @@
 using System;
 using System.Collections.Generic;
 using System.Linq;
+using System.Linq.Expressions;
 using System.Reflection;
 using System.Threading;
 
@@ -162,7 +163,7 @@ public void CreateMarshalFromJniMethodExpression_InstanceAction ()
 		{
 			var t = typeof (ExportTest);
 			var m = t.GetMethod ("InstanceAction");
-			CheckCreateInvocationExpression (null, t, m, typeof (Action<IntPtr, IntPtr>),
+			CheckCreateMarshalToManagedExpression (null, t, m, typeof (Action<IntPtr, IntPtr>),
 					@"void (IntPtr __jnienv, IntPtr __this)
 {
 	JniTransition __envp;
@@ -188,28 +189,33 @@ public void CreateMarshalFromJniMethodExpression_InstanceAction ()
 }");
 		}
 
-		static void CheckCreateInvocationExpression (JavaCallableAttribute export, Type type, MethodInfo method, Type expectedDelegateType, string expectedBody)
+		static void CheckCreateMarshalToManagedExpression (JavaCallableAttribute export, Type type, MethodInfo method, Type expectedDelegateType, string expectedBody)
 		{
-			export  = export ?? new JavaCallableAttribute ();
-			var b   = CreateBuilder ();
-			var l   = b.CreateMarshalToManagedExpression (method, export, type);
-			Console.WriteLine ("## method: {0}", method.Name);
-			Console.WriteLine (l.ToCSharpCode ());
+			export = export ?? new JavaCallableAttribute ();
+			var b = CreateBuilder ();
+			var l = b.CreateMarshalToManagedExpression (method, export, type);
+			CheckExpression (l, method.Name, expectedDelegateType, expectedBody);
+		}
+
+		static void CheckExpression (LambdaExpression expression, string memberName, Type expressionType, string expectedBody)
+		{
+			Console.WriteLine ("## member: {0}", memberName);
+			Console.WriteLine (expression.ToCSharpCode ());
 			var da = AppDomain.CurrentDomain.DefineDynamicAssembly(
 				new AssemblyName("dyn"), // call it whatever you want
 				System.Reflection.Emit.AssemblyBuilderAccess.Save);
 
-			var _name = "dyn-" + method.Name + ".dll";
+			var _name = "dyn-" + memberName + ".dll";
 			var dm = da.DefineDynamicModule("dyn_mod", _name);
 			var dt = dm.DefineType ("dyn_type", TypeAttributes.Public);
 			var mb = dt.DefineMethod(
-				method.Name,
+				memberName,
 				MethodAttributes.Public | MethodAttributes.Static);
 
-			l.CompileToMethod(mb);
+			expression.CompileToMethod (mb);
 			dt.CreateType();
-			Assert.AreEqual (expectedDelegateType, l.Type);
-			Assert.AreEqual (expectedBody, l.ToCSharpCode ());
+			Assert.AreEqual (expressionType,    expression.Type);
+			Assert.AreEqual (expectedBody,      expression.ToCSharpCode ());
 #if !__ANDROID__
 			da.Save (_name);
 #endif  // !__ANDROID__
@@ -220,7 +226,7 @@ public void CreateMarshalFromJniMethodExpression_StaticAction ()
 		{
 			var t       = typeof (ExportTest);
 			Action a    = ExportTest.StaticAction;
-			CheckCreateInvocationExpression (null, t, a.Method, typeof(Action<IntPtr, IntPtr>),
+			CheckCreateMarshalToManagedExpression (null, t, a.Method, typeof(Action<IntPtr, IntPtr>),
 					@"void (IntPtr __jnienv, IntPtr __class)
 {
 	JniTransition __envp;
@@ -252,7 +258,7 @@ public void CreateMarshalFromJniMethodExpression_StaticActionInt32String ()
 			var e = new JavaCallableAttribute () {
 				Signature = "(ILjava/lang/String;)V",
 			};
-			CheckCreateInvocationExpression (e, t, m.Method, typeof (Action<IntPtr, IntPtr, int, IntPtr>),
+			CheckCreateMarshalToManagedExpression (e, t, m.Method, typeof (Action<IntPtr, IntPtr, int, IntPtr>),
 					@"void (IntPtr __jnienv, IntPtr __class, int i, IntPtr v)
 {
 	JniTransition __envp;
@@ -286,7 +292,7 @@ public void CreateMarshalFromJniMethodExpression_StaticFuncMyLegacyColorMyColor_
 			var e = new JavaCallableAttribute () {
 				Signature = "(II)I",
 			};
-			CheckCreateInvocationExpression (e, t, m.Method, typeof (Func<IntPtr, IntPtr, int, int, int>),
+			CheckCreateMarshalToManagedExpression (e, t, m.Method, typeof (Func<IntPtr, IntPtr, int, int, int>),
 					@"int (IntPtr __jnienv, IntPtr __class, int color1, int color2)
 {
 	JniTransition __envp;
@@ -327,7 +333,7 @@ public void CreateMarshalFromJniMethodExpression_FuncInt64 ()
 			var e = new JavaCallableAttribute () {
 				Signature = "()J",
 			};
-			CheckCreateInvocationExpression (e, t, m, typeof (Func<IntPtr, IntPtr, long>),
+			CheckCreateMarshalToManagedExpression (e, t, m, typeof (Func<IntPtr, IntPtr, long>),
 					@"long (IntPtr __jnienv, IntPtr __this)
 {
 	JniTransition __envp;
@@ -364,7 +370,7 @@ public void CreateMarshalFromJniMethodExpression_FuncIJavaObject ()
 			var e = new JavaCallableAttribute () {
 				Signature = "()Ljava/lang/Object;",
 			};
-			CheckCreateInvocationExpression (e, t, m, typeof (Func<IntPtr, IntPtr, IntPtr>),
+			CheckCreateMarshalToManagedExpression (e, t, m, typeof (Func<IntPtr, IntPtr, IntPtr>),
 					@"IntPtr (IntPtr __jnienv, IntPtr __this)
 {
 	JniTransition __envp;
@@ -418,7 +424,7 @@ public void CreateMarshalToManagedExpression_DirectMethod ()
 			var e = new JavaCallableAttribute () {
 				Signature = "()V",
 			};
-			CheckCreateInvocationExpression (e, a.Method.DeclaringType, a.Method, typeof (Action<IntPtr, IntPtr>),
+			CheckCreateMarshalToManagedExpression (e, a.Method.DeclaringType, a.Method, typeof (Action<IntPtr, IntPtr>),
 				@"void (IntPtr jnienv, IntPtr context)
 {
 	JniTransition __envp;
@@ -441,5 +447,34 @@ public void CreateMarshalToManagedExpression_DirectMethod ()
 	}
 }");
 		}
-			}
+
+		[Test]
+		public void CreateConstructActivationPeerExpression_Exceptions ()
+		{
+			var b   = CreateBuilder ();
+			Assert.Throws<ArgumentNullException> (() => b.CreateConstructActivationPeerExpression (null));
+		}
+
+		[Test]
+		public void CreateConstructActivationPeerExpression ()
+		{
+			var b   = CreateBuilder ();
+			var c   = typeof (ExportedMemberBuilderTest).GetConstructor (new Type [0]);
+			var e   = b.CreateConstructActivationPeerExpression (c);
+			CheckExpression (e,
+					"ExportedMemberBuilderTest_ctor",
+					typeof(Func<ConstructorInfo, JniObjectReference, object[], object>),
+					@"object (ConstructorInfo constructor, JniObjectReference reference, object[] parameters)
+{
+	Type type;
+	object self;
+
+	type = constructor.DeclaringType;
+	self = FormatterServices.GetUninitializedObject(type);
+	(IJavaPeerable)self.SetPeerReference(reference);
+	constructor.Invoke(self, parameters);
+	return self;
+}");
+		}
+	}
 }

src/Java.Interop/Java.Interop/JavaException.cs

@@ -29,7 +29,11 @@ unsafe public class JavaException : Exception, IJavaPeerable
 		protected   static  readonly    JniObjectReference*     InvalidJniObjectReference = null;
 
 		public unsafe JavaException ()
+			: this (ref *InvalidJniObjectReference, JniObjectReferenceOptions.None)
 		{
+			if (PeerReference.IsValid)
+				return;
+
 			var peer = JniPeerMembers.InstanceMethods.StartCreateInstance ("()V", GetType (), null);
 			Construct (ref peer, JniObjectReferenceOptions.CopyAndDispose);
 			JniPeerMembers.InstanceMethods.FinishCreateInstance ("()V", this, null);
@@ -73,14 +77,9 @@ public unsafe JavaException (string message, Exception innerException)
 		public JavaException (ref JniObjectReference reference, JniObjectReferenceOptions transfer)
 			: base (_GetMessage (ref reference, transfer), _GetCause (ref reference, transfer))
 		{
-			if (transfer == JniObjectReferenceOptions.None)
-				return;
-
-			if (!reference.IsValid)
-				return;
-
 			Construct (ref reference, transfer);
-			javaStackTrace    = _GetJavaStack (PeerReference);
+			if (PeerReference.IsValid)
+				javaStackTrace    = _GetJavaStack (PeerReference);
 		}
 
 		protected void Construct (ref JniObjectReference reference, JniObjectReferenceOptions options)

src/Java.Interop/Java.Interop/JavaObject.cs

@@ -59,14 +59,15 @@ public  virtual JniPeerMembers              JniPeerMembers {
 
 		public JavaObject (ref JniObjectReference reference, JniObjectReferenceOptions options)
 		{
-			if (options == JniObjectReferenceOptions.None)
-				return;
-
 			Construct (ref reference, options);
 		}
 
 		public unsafe JavaObject ()
+			: this (ref *InvalidJniObjectReference, JniObjectReferenceOptions.None)
 		{
+			if (PeerReference.IsValid)
+				return;
+
 			var peer = JniPeerMembers.InstanceMethods.StartCreateInstance ("()V", GetType (), null);
 			Construct (ref peer, JniObjectReferenceOptions.CopyAndDispose);
 			JniPeerMembers.InstanceMethods.FinishCreateInstance ("()V", this, null);

src/Java.Interop/Java.Interop/JniObjectReference.cs

@@ -197,6 +197,9 @@ public static void Dispose (ref JniObjectReference reference)
 
 		public static void Dispose (ref JniObjectReference reference, JniObjectReferenceOptions options)
 		{
+			if (options == JniObjectReferenceOptions.None)
+				return;
+
 			if (!reference.IsValid)
 				return;
 

src/Java.Interop/Java.Interop/JniRuntime.JniExportedMemberBuilder.cs

@@ -62,6 +62,17 @@ public  Delegate                                                CreateMarshalToM
 
 			public  abstract    LambdaExpression                            CreateMarshalToManagedExpression (MethodInfo method);
 			public  abstract    IEnumerable<JniNativeMethodRegistration>    GetExportedMemberRegistrations (Type declaringType);
+
+			public  abstract    Expression<Func<ConstructorInfo, JniObjectReference, object[], object>>     CreateConstructActivationPeerExpression (ConstructorInfo constructor);
+
+			public  Func<ConstructorInfo, JniObjectReference, object[], object>                             CreateConstructActivationPeerFunc (ConstructorInfo constructor)
+			{
+				if (constructor == null)
+					throw new ArgumentNullException (nameof (constructor));
+
+				var e   = CreateConstructActivationPeerExpression (constructor);
+				return e.Compile ();
+			}
 		}
 	}
 }

src/Java.Interop/Java.Interop/JniRuntime.JniValueManager.cs

@@ -72,43 +72,47 @@ protected virtual void Dispose (bool disposing)
 
 			public abstract List<JniSurfacedPeerInfo>   GetSurfacedPeers ();
 
-			public void Construct (IJavaPeerable value, ref JniObjectReference reference, JniObjectReferenceOptions options)
+			public void Construct (IJavaPeerable peer, ref JniObjectReference reference, JniObjectReferenceOptions options)
 			{
-				if (value == null)
-					throw new ArgumentNullException (nameof (value));
+				if (peer == null)
+					throw new ArgumentNullException (nameof (peer));
 
-				var activationRef   = value.PeerReference;
-				if (activationRef.IsValid) {
-					value.SetJniManagedPeerState (value.JniManagedPeerState | JniManagedPeerStates.Activatable);
+				var newRef  = peer.PeerReference;
+				if (newRef.IsValid) {
+					// Activation! See ManagedPeer.RunConstructor
+					peer.SetJniManagedPeerState (peer.JniManagedPeerState | JniManagedPeerStates.Activatable);
 					JniObjectReference.Dispose (ref reference, options);
-					reference   = activationRef;
-					options     = JniObjectReferenceOptions.Copy;
-				}
+					newRef   = newRef.NewGlobalRef ();
+				} else if (options == JniObjectReferenceOptions.None) {
+					// `reference` is likely *InvalidJniObjectReference, and can't be touched
+					return;
+				} else if (!reference.IsValid) {
+					throw new ArgumentException ("JNI Object Reference is invalid.", nameof (reference));
+				} else {
+					newRef  = reference;
 
-				if (!reference.IsValid)
-					throw new ArgumentException ("handle is invalid.", nameof (reference));
+					if ((options & JniObjectReferenceOptions.Copy) == JniObjectReferenceOptions.Copy) {
+						newRef  = reference.NewGlobalRef ();
+					}
 
-				var newRef      = reference;
-				if ((options & JniObjectReferenceOptions.Copy) == JniObjectReferenceOptions.Copy) {
-					newRef  = reference.NewGlobalRef ();
+					JniObjectReference.Dispose (ref reference, options);
 				}
-				value.SetPeerReference (newRef);
-				JniObjectReference.Dispose (ref reference, options);
 
-				value.SetJniIdentityHashCode (JniSystem.IdentityHashCode (newRef));
+				peer.SetPeerReference (newRef);
+				peer.SetJniIdentityHashCode (JniSystem.IdentityHashCode (newRef));
 
 				var o = Runtime.ObjectReferenceManager;
 				if (o.LogGlobalReferenceMessages) {
 					o.WriteGlobalReferenceLine ("Created PeerReference={0} IdentityHashCode=0x{1} Instance=0x{2} Instance.Type={3}, Java.Type={4}",
 							newRef.ToString (),
-							value.JniIdentityHashCode.ToString ("x"),
-							RuntimeHelpers.GetHashCode (value).ToString ("x"),
-							value.GetType ().FullName,
+							peer.JniIdentityHashCode.ToString ("x"),
+							RuntimeHelpers.GetHashCode (peer).ToString ("x"),
+							peer.GetType ().FullName,
 							JniEnvironment.Types.GetJniTypeNameFromInstance (newRef));
 				}
 
 				if ((options & DoNotRegisterTarget) != DoNotRegisterTarget) {
-					Add (value);
+					Add (peer);
 				}
 			}