Skip to content

[generator] Support Kotlin's unsigned types (#539) #553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 7, 2020
Merged

Conversation

jpobst
Copy link
Contributor

@jpobst jpobst commented Jan 7, 2020

Fixes: #525

Context: dotnet/android#4054
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Another place where Kotlin makes use of "name mangling" -- see also
commit f3553f4 -- is in the use of unsigned types such as UInt.
At the JVM ABI level, Kotlin treats unsigned types as their signed
counterparts, e.g. kotlin.UInt is an int and kotlin.UIntArray
is an int[]:

// Kotlin
public class Example {
  public fun value(value: UInt) : UInt {
    return value
  }
  public fun array(value: UIntArray) : UIntArray {
    return value
  }
}

// `javap` output:
public final class Example {
  public final int value-WZ4Q5Ns(int);
  public final int[] array--ajY-9A(int[]);
}

Kotlin uses Java Annotations to determine whether a parameter or
return type is actually an unsigned type instead of a signed type.

Update Xamarin.Android.Tools.Bytecode and generator to bind e.g.:

  • kotlin.UInt as System.UInt32
  • kotlin.UIntArray as a System.UInt32[]

and likewise for the other unsigned types ushort, ulong, ubyte.

In order to do this, we pretend that they are native Java types and
just translate a few places where we need to tell Java the real type.

~~ Xamarin.Android.Tools.Bytecode / class-parse ~~

When we read the Kotlin metadata in the Java bytecode, if we come
across one of these types we store it within an additional
FieldInfo.KotlinType property that we can access later. When we
are generating the XML we check this additional flag and if it's one
of our types we emit it instead of the native Java type.

For example:

<method
  abstract="false"
  deprecated="not deprecated"
  final="false"
  name="unsignedAbstractMethod-WZ4Q5Ns"
  native="false"
  return="uint"
  jni-return="I"
  static="false"
  synchronized="false"
  visibility="public"
  bridge="false"
  synthetic="false"
  jni-signature="(I)I">
  <parameter
    name="value"
    type="uint"
    jni-type="I" />
</method>

Here we see that even though @jni-return is I -- meaning int --
the @return property is uint. Likewise parameter/@jni-type and
parameter/@type. The JNI ABI is int, but we bind in C# as uint.

~~ ApiXmlAdjuster ~~

Update JavaTypeReference to contain unsigned types:

UInt   = new JavaTypeReference ("uint");
UShort = new JavaTypeReference ("ushort");
ULong  = new JavaTypeReference ("ulong");
UByte  = new JavaTypeReference ("ubyte");

~~ generator ~~

generator has the 4 new types added to the SymbolTable as
SimpleSymbols:

AddType (new SimpleSymbol ("0", "uint",    "uint",   "I", returnCast: "(uint)"));
AddType (new SimpleSymbol ("0", "ushort",  "ushort", "S", returnCast: "(ushort)"));
AddType (new SimpleSymbol ("0", "ulong",   "ulong",  "J", returnCast: "(ulong)"));
AddType (new SimpleSymbol ("0", "ubyte",   "byte",   "B", returnCast: "(byte)"));

There are 2 fixups we have to make because we use GetIntValue(...),
etc. instead of having unsigned versions:

  • Override name of which method to call, e.g.: GetIntValue()
    instead of GetUintValue().
  • Cast the int value returned to uint. This is accomplished
    via the new ISymbol.ReturnCast property.

~~ A Note On API Compatibility ~~

Bindings which use Kotlin Unsigned Types will only work on
Xamarin.Android 10.2.0 or later ("Visual Studio 16.5").

The problem is that while we can emit C# source code which will
compile against older versions of Xamarin.Android, if they use
arrays they will not run under older versions of Xamarin.Android.

For example, imagine this binding code for the above Kotlin
Example.array() method:

// C# Binding of Example.array()
partial class Example {
  public unsafe uint[] Array(uint[] value)
  {
    const string      __id          = "array--ajY-9A.([I)[I";
    IntPtr            native_value  = JNIEnv.NewArray ((int[]) (object) value); // Works!...ish?
    JniArgumentValue* __args        = stackalloc JniArgumentValue [1];
    __args [0]                      = new JniArgumentValue (native_value);

    JniObjectReference r            = _members.InstanceMethods.InvokeVirtualIntMethod (__id, this, __args);
    return (uint[]) JNIEnv.GetArray (r.Handle, JniHandleOwnership.DoNotTransfer, typeof (uint));
  }
}

That could conceivably compile against older Xamarin.Android
versions. However, that cannot run against older Xamarin.Android
versions, as eventually JNIEnv.GetArray() will hit some
dictionaries to determine how to marshal IntPtr to a uint[], at
which point things will fail because there is no such mapping until
Xamarin.Android 10.2.0.

We feel that a "hard" ABI requirement will have more "graceful"
failure conditions than a solution which doesn't add ABI requirements.
In this case, if you create a Kotlin library binding which exposes
unsigned types, attempting to build an app in Release configuration
against older Xamarin.Android versions will result in a linker error,
as the required JNIEnv methods will not be resolvable.

(cherry picked from commit 71afce5)

@jonpryor
Copy link
Contributor

jonpryor commented Jan 7, 2020

/azp run

@azure-pipelines
Copy link

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

Fixes: #525

Context: dotnet/android#4054
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Another place where Kotlin makes use of "name mangling" -- see also
commit f3553f4 -- is in the use of unsigned types such as `UInt`.
At the JVM ABI level, Kotlin treats unsigned types as their signed
counterparts, e.g. `kotlin.UInt` is an `int` and `kotlin.UIntArray`
is an `int[]`:

	// Kotlin
	public class Example {
	  public fun value(value: UInt) : UInt {
	    return value
	  }
	  public fun array(value: UIntArray) : UIntArray {
	    return value
	  }
	}

	// `javap` output:
	public final class Example {
	  public final int value-WZ4Q5Ns(int);
	  public final int[] array--ajY-9A(int[]);
	}

Kotlin uses Java Annotations to determine whether a parameter or
return type is actually an unsigned type instead of a signed type.

Update `Xamarin.Android.Tools.Bytecode` and `generator` to bind e.g.:

  * `kotlin.UInt` as `System.UInt32`
  * `kotlin.UIntArray` as a `System.UInt32[]`

and likewise for the other unsigned types `ushort`, `ulong`, `ubyte`.

In order to do this, we pretend that they are native Java types and
just translate a few places where we need to tell Java the real type.

~~ Xamarin.Android.Tools.Bytecode / class-parse ~~

When we read the Kotlin metadata in the Java bytecode, if we come
across one of these types we store it within an additional
`FieldInfo.KotlinType` property that we can access later.  When we
are generating the XML we check this additional flag and if it's one
of our types we emit it instead of the native Java type.

For example:

	<method
	  abstract="false"
	  deprecated="not deprecated"
	  final="false"
	  name="unsignedAbstractMethod-WZ4Q5Ns"
	  native="false"
	  return="uint"
	  jni-return="I"
	  static="false"
	  synchronized="false"
	  visibility="public"
	  bridge="false"
	  synthetic="false"
	  jni-signature="(I)I">
	  <parameter
	    name="value"
	    type="uint"
	    jni-type="I" />
	</method>

Here we see that even though `@jni-return` is `I` -- meaning `int` --
the `@return` property is `uint`.  Likewise `parameter/@jni-type` and
`parameter/@type`.  The JNI ABI is `int`, but we bind in C# as `uint`.

~~ ApiXmlAdjuster ~~

Update `JavaTypeReference` to contain unsigned types:

	UInt   = new JavaTypeReference ("uint");
	UShort = new JavaTypeReference ("ushort");
	ULong  = new JavaTypeReference ("ulong");
	UByte  = new JavaTypeReference ("ubyte");

~~ generator ~~

`generator` has the 4 new types added to the `SymbolTable` as
`SimpleSymbols`:

	AddType (new SimpleSymbol ("0", "uint",    "uint",   "I", returnCast: "(uint)"));
	AddType (new SimpleSymbol ("0", "ushort",  "ushort", "S", returnCast: "(ushort)"));
	AddType (new SimpleSymbol ("0", "ulong",   "ulong",  "J", returnCast: "(ulong)"));
	AddType (new SimpleSymbol ("0", "ubyte",   "byte",   "B", returnCast: "(byte)"));

There are 2 fixups we have to make because we use `GetIntValue(...)`,
etc. instead of having unsigned versions:

  * Override name of which method to call, e.g.: `GetIntValue()`
    instead of `GetUintValue()`.
  * Cast the `int` value returned to `uint`.  This is accomplished
    via the new `ISymbol.ReturnCast` property.

~~ A Note On API Compatibility ~~

Bindings which use Kotlin Unsigned Types will *only* work on
Xamarin.Android 10.2.0 or later ("Visual Studio 16.5").

The problem is that while we *can* emit C# source code which will
*compile* against older versions of Xamarin.Android, if they use
arrays they will not *run* under older versions of Xamarin.Android.

For example, imagine this binding code for the above Kotlin
`Example.array()` method:

	// C# Binding of Example.array()
	partial class Example {
	  public unsafe uint[] Array(uint[] value)
	  {
	    const string      __id          = "array--ajY-9A.([I)[I";
	    IntPtr            native_value  = JNIEnv.NewArray ((int[]) (object) value); // Works!...ish?
	    JniArgumentValue* __args        = stackalloc JniArgumentValue [1];
	    __args [0]                      = new JniArgumentValue (native_value);

	    JniObjectReference r            = _members.InstanceMethods.InvokeVirtualIntMethod (__id, this, __args);
	    return (uint[]) JNIEnv.GetArray (r.Handle, JniHandleOwnership.DoNotTransfer, typeof (uint));
	  }
	}

That could conceivably *compile* against older Xamarin.Android
versions.  However, that cannot *run* against older Xamarin.Android
versions, as *eventually* `JNIEnv.GetArray()` will hit some
dictionaries to determine how to marshal `IntPtr` to a `uint[]`, at
which point things will fail because there is no such mapping *until*
Xamarin.Android 10.2.0.

We feel that a "hard" ABI requirement will have more "graceful"
failure conditions than a solution which doesn't add ABI requirements.
In this case, if you create a Kotlin library binding which exposes
unsigned types, attempting to build an app in Release configuration
against older Xamarin.Android versions will result in a linker error,
as the required `JNIEnv` methods will not be resolvable.

(cherry picked from commit 71afce5)
@jonpryor jonpryor merged commit c0cc770 into d16-5 Jan 7, 2020
@jonpryor jonpryor deleted the kotlin-cp branch January 7, 2020 22:03
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants