Skip to content

[JNIEnv] Add support for Kotlin's unsigned types. #4054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 8, 2020
Merged

[JNIEnv] Add support for Kotlin's unsigned types. #4054

merged 1 commit into from
Jan 8, 2020

Conversation

jpobst
Copy link
Contributor

@jpobst jpobst commented Dec 18, 2019 •
edited
Loading

Companion to dotnet/java-interop#539.

  • There were changes need to JNIEnv to support marshaling unsigned types for arrays.
  • Add protobuf-net.dll to our installers, as it is required to read the metadata Kotlin puts in jar files.
  • Add on-device tests for calling Kotlin code that uses unsigned types.
    • We don't currently have a way of acquiring the Kotlin compiler for builds, so the test .jar and the Kotlin stdlib .jar are included. The source of the test .jar is provided for future use.

@jpobst jpobst force-pushed the unsigned-types branch 3 times, most recently from 68b33f8 to 4c2f8ef Compare December 19, 2019 16:44
@jpobst jpobst mentioned this pull request Dec 19, 2019
Merged
@jpobst jpobst marked this pull request as ready for review December 19, 2019 20:14
jonpryor pushed a commit to dotnet/java-interop that referenced this pull request Jan 7, 2020
@jpobst @jonpryor
[generator] Support Kotlin's unsigned types (#539)
71afce5 
Fixes: #525

Context: dotnet/android#4054
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Another place where Kotlin makes use of "name mangling" -- see also
commit f3553f4 -- is in the use of unsigned types such as `UInt`.
At the JVM ABI level, Kotlin treats unsigned types as their signed
counterparts, e.g. `kotlin.UInt` is an `int` and `kotlin.UIntArray`
is an `int[]`:

	// Kotlin
	public class Example {
	  public fun value(value: UInt) : UInt {
	    return value
	  }
	  public fun array(value: UIntArray) : UIntArray {
	    return value
	  }
	}

	// `javap` output:
	public final class Example {
	  public final int value-WZ4Q5Ns(int);
	  public final int[] array--ajY-9A(int[]);
	}	

Kotlin uses Java Annotations to determine whether a parameter or
return type is actually an unsigned type instead of a signed type.

Update `Xamarin.Android.Tools.Bytecode` and `generator` to bind e.g.:

  * `kotlin.UInt` as `System.UInt32`
  * `kotlin.UIntArray` as a `System.UInt32[]`

and likewise for the other unsigned types `ushort`, `ulong`, `ubyte`.

In order to do this, we pretend that they are native Java types and
just translate a few places where we need to tell Java the real type.

~~ Xamarin.Android.Tools.Bytecode / class-parse ~~

When we read the Kotlin metadata in the Java bytecode, if we come
across one of these types we store it within an additional
`FieldInfo.KotlinType` property that we can access later.  When we
are generating the XML we check this additional flag and if it's one
of our types we emit it instead of the native Java type.

For example:

	<method
	  abstract="false"
	  deprecated="not deprecated"
	  final="false"
	  name="unsignedAbstractMethod-WZ4Q5Ns"
	  native="false"
	  return="uint"
	  jni-return="I"
	  static="false"
	  synchronized="false"
	  visibility="public"
	  bridge="false"
	  synthetic="false"
	  jni-signature="(I)I">
	  <parameter
	    name="value"
	    type="uint"
	    jni-type="I" />
	</method>

Here we see that even though `@jni-return` is `I` -- meaning `int` --
the `@return` property is `uint`.  Likewise `parameter/@jni-type` and
`parameter/@type`.  The JNI ABI is `int`, but we bind in C# as `uint`.

~~ ApiXmlAdjuster ~~

Update `JavaTypeReference` to contain unsigned types:

	UInt   = new JavaTypeReference ("uint");
	UShort = new JavaTypeReference ("ushort");
	ULong  = new JavaTypeReference ("ulong");
	UByte  = new JavaTypeReference ("ubyte");

~~ generator ~~

`generator` has the 4 new types added to the `SymbolTable` as
`SimpleSymbols`:

	AddType (new SimpleSymbol ("0", "uint",    "uint",   "I", returnCast: "(uint)"));
	AddType (new SimpleSymbol ("0", "ushort",  "ushort", "S", returnCast: "(ushort)"));
	AddType (new SimpleSymbol ("0", "ulong",   "ulong",  "J", returnCast: "(ulong)"));
	AddType (new SimpleSymbol ("0", "ubyte",   "byte",   "B", returnCast: "(byte)"));

There are 2 fixups we have to make because we use `GetIntValue(...)`,
etc. instead of having unsigned versions:

  * Override name of which method to call, e.g.: `GetIntValue()`
    instead of `GetUintValue()`.
  * Cast the `int` value returned to `uint`.  This is accomplished
    via the new `ISymbol.ReturnCast` property.

~~ A Note On API Compatibility ~~

Bindings which use Kotlin Unsigned Types will *only* work on
Xamarin.Android 10.2.0 or later ("Visual Studio 16.5").

The problem is that while we *can* emit C# source code which will
*compile* against older versions of Xamarin.Android, if they use
arrays they will not *run* under older versions of Xamarin.Android.

For example, imagine this binding code for the above Kotlin
`Example.array()` method:

	// C# Binding of Example.array()
	partial class Example {
	  public unsafe uint[] Array(uint[] value)
	  {
	    const string      __id          = "array--ajY-9A.([I)[I";
	    IntPtr            native_value  = JNIEnv.NewArray ((int[]) (object) value); // Works!...ish?
	    JniArgumentValue* __args        = stackalloc JniArgumentValue [1];
	    __args [0]                      = new JniArgumentValue (native_value);

	    JniObjectReference r            = _members.InstanceMethods.InvokeVirtualIntMethod (__id, this, __args);
	    return (uint[]) JNIEnv.GetArray (r.Handle, JniHandleOwnership.DoNotTransfer, typeof (uint));
	  }
	}

That could conceivably *compile* against older Xamarin.Android
versions.  However, that cannot *run* against older Xamarin.Android
versions, as *eventually* `JNIEnv.GetArray()` will hit some
dictionaries to determine how to marshal `IntPtr` to a `uint[]`, at
which point things will fail because there is no such mapping *until*
Xamarin.Android 10.2.0.

We feel that a "hard" ABI requirement will have more "graceful"
failure conditions than a solution which doesn't add ABI requirements.
In this case, if you create a Kotlin library binding which exposes
unsigned types, attempting to build an app in Release configuration
against older Xamarin.Android versions will result in a linker error,
as the required `JNIEnv` methods will not be resolvable.
@jonpryor
Copy link
Contributor

jonpryor commented Jan 7, 2020

While thinking through the eventual commit message for dotnet/java-interop@71afce5 , it occurred to me that we might also need JNIEnv.NewArray() method overloads as well to take uint[]/etc. types.

Has a complete on-device end-to-end test been performed with Kotlin unsigned types, in particular array types?

jpobst added a commit to dotnet/java-interop that referenced this pull request Jan 7, 2020
@jpobst
[generator] Support Kotlin's unsigned types (#539)
e8dfea1 
Fixes: #525

Context: dotnet/android#4054
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Another place where Kotlin makes use of "name mangling" -- see also
commit f3553f4 -- is in the use of unsigned types such as `UInt`.
At the JVM ABI level, Kotlin treats unsigned types as their signed
counterparts, e.g. `kotlin.UInt` is an `int` and `kotlin.UIntArray`
is an `int[]`:

	// Kotlin
	public class Example {
	  public fun value(value: UInt) : UInt {
	    return value
	  }
	  public fun array(value: UIntArray) : UIntArray {
	    return value
	  }
	}

	// `javap` output:
	public final class Example {
	  public final int value-WZ4Q5Ns(int);
	  public final int[] array--ajY-9A(int[]);
	}

Kotlin uses Java Annotations to determine whether a parameter or
return type is actually an unsigned type instead of a signed type.

Update `Xamarin.Android.Tools.Bytecode` and `generator` to bind e.g.:

  * `kotlin.UInt` as `System.UInt32`
  * `kotlin.UIntArray` as a `System.UInt32[]`

and likewise for the other unsigned types `ushort`, `ulong`, `ubyte`.

In order to do this, we pretend that they are native Java types and
just translate a few places where we need to tell Java the real type.

~~ Xamarin.Android.Tools.Bytecode / class-parse ~~

When we read the Kotlin metadata in the Java bytecode, if we come
across one of these types we store it within an additional
`FieldInfo.KotlinType` property that we can access later.  When we
are generating the XML we check this additional flag and if it's one
of our types we emit it instead of the native Java type.

For example:

	<method
	  abstract="false"
	  deprecated="not deprecated"
	  final="false"
	  name="unsignedAbstractMethod-WZ4Q5Ns"
	  native="false"
	  return="uint"
	  jni-return="I"
	  static="false"
	  synchronized="false"
	  visibility="public"
	  bridge="false"
	  synthetic="false"
	  jni-signature="(I)I">
	  <parameter
	    name="value"
	    type="uint"
	    jni-type="I" />
	</method>

Here we see that even though `@jni-return` is `I` -- meaning `int` --
the `@return` property is `uint`.  Likewise `parameter/@jni-type` and
`parameter/@type`.  The JNI ABI is `int`, but we bind in C# as `uint`.

~~ ApiXmlAdjuster ~~

Update `JavaTypeReference` to contain unsigned types:

	UInt   = new JavaTypeReference ("uint");
	UShort = new JavaTypeReference ("ushort");
	ULong  = new JavaTypeReference ("ulong");
	UByte  = new JavaTypeReference ("ubyte");

~~ generator ~~

`generator` has the 4 new types added to the `SymbolTable` as
`SimpleSymbols`:

	AddType (new SimpleSymbol ("0", "uint",    "uint",   "I", returnCast: "(uint)"));
	AddType (new SimpleSymbol ("0", "ushort",  "ushort", "S", returnCast: "(ushort)"));
	AddType (new SimpleSymbol ("0", "ulong",   "ulong",  "J", returnCast: "(ulong)"));
	AddType (new SimpleSymbol ("0", "ubyte",   "byte",   "B", returnCast: "(byte)"));

There are 2 fixups we have to make because we use `GetIntValue(...)`,
etc. instead of having unsigned versions:

  * Override name of which method to call, e.g.: `GetIntValue()`
    instead of `GetUintValue()`.
  * Cast the `int` value returned to `uint`.  This is accomplished
    via the new `ISymbol.ReturnCast` property.

~~ A Note On API Compatibility ~~

Bindings which use Kotlin Unsigned Types will *only* work on
Xamarin.Android 10.2.0 or later ("Visual Studio 16.5").

The problem is that while we *can* emit C# source code which will
*compile* against older versions of Xamarin.Android, if they use
arrays they will not *run* under older versions of Xamarin.Android.

For example, imagine this binding code for the above Kotlin
`Example.array()` method:

	// C# Binding of Example.array()
	partial class Example {
	  public unsafe uint[] Array(uint[] value)
	  {
	    const string      __id          = "array--ajY-9A.([I)[I";
	    IntPtr            native_value  = JNIEnv.NewArray ((int[]) (object) value); // Works!...ish?
	    JniArgumentValue* __args        = stackalloc JniArgumentValue [1];
	    __args [0]                      = new JniArgumentValue (native_value);

	    JniObjectReference r            = _members.InstanceMethods.InvokeVirtualIntMethod (__id, this, __args);
	    return (uint[]) JNIEnv.GetArray (r.Handle, JniHandleOwnership.DoNotTransfer, typeof (uint));
	  }
	}

That could conceivably *compile* against older Xamarin.Android
versions.  However, that cannot *run* against older Xamarin.Android
versions, as *eventually* `JNIEnv.GetArray()` will hit some
dictionaries to determine how to marshal `IntPtr` to a `uint[]`, at
which point things will fail because there is no such mapping *until*
Xamarin.Android 10.2.0.

We feel that a "hard" ABI requirement will have more "graceful"
failure conditions than a solution which doesn't add ABI requirements.
In this case, if you create a Kotlin library binding which exposes
unsigned types, attempting to build an app in Release configuration
against older Xamarin.Android versions will result in a linker error,
as the required `JNIEnv` methods will not be resolvable.

(cherry picked from commit 71afce5)
@jpobst jpobst mentioned this pull request Jan 7, 2020
Merged
@jpobst
Copy link
Contributor Author

jpobst commented Jan 7, 2020

This PR includes on-device tests that roundtrip the various unsigned array types:
https://github.com/xamarin/xamarin-android/pull/4054/files#diff-e5a7b7515df9a54919e687e8c196a0b1R52

Is that what you are looking for?

jpobst added a commit to dotnet/java-interop that referenced this pull request Jan 7, 2020
@jpobst
[generator] Support Kotlin's unsigned types (#539)
1c0d7a5 
Fixes: #525

Context: dotnet/android#4054
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Another place where Kotlin makes use of "name mangling" -- see also
commit f3553f4 -- is in the use of unsigned types such as `UInt`.
At the JVM ABI level, Kotlin treats unsigned types as their signed
counterparts, e.g. `kotlin.UInt` is an `int` and `kotlin.UIntArray`
is an `int[]`:

	// Kotlin
	public class Example {
	  public fun value(value: UInt) : UInt {
	    return value
	  }
	  public fun array(value: UIntArray) : UIntArray {
	    return value
	  }
	}

	// `javap` output:
	public final class Example {
	  public final int value-WZ4Q5Ns(int);
	  public final int[] array--ajY-9A(int[]);
	}

Kotlin uses Java Annotations to determine whether a parameter or
return type is actually an unsigned type instead of a signed type.

Update `Xamarin.Android.Tools.Bytecode` and `generator` to bind e.g.:

  * `kotlin.UInt` as `System.UInt32`
  * `kotlin.UIntArray` as a `System.UInt32[]`

and likewise for the other unsigned types `ushort`, `ulong`, `ubyte`.

In order to do this, we pretend that they are native Java types and
just translate a few places where we need to tell Java the real type.

~~ Xamarin.Android.Tools.Bytecode / class-parse ~~

When we read the Kotlin metadata in the Java bytecode, if we come
across one of these types we store it within an additional
`FieldInfo.KotlinType` property that we can access later.  When we
are generating the XML we check this additional flag and if it's one
of our types we emit it instead of the native Java type.

For example:

	<method
	  abstract="false"
	  deprecated="not deprecated"
	  final="false"
	  name="unsignedAbstractMethod-WZ4Q5Ns"
	  native="false"
	  return="uint"
	  jni-return="I"
	  static="false"
	  synchronized="false"
	  visibility="public"
	  bridge="false"
	  synthetic="false"
	  jni-signature="(I)I">
	  <parameter
	    name="value"
	    type="uint"
	    jni-type="I" />
	</method>

Here we see that even though `@jni-return` is `I` -- meaning `int` --
the `@return` property is `uint`.  Likewise `parameter/@jni-type` and
`parameter/@type`.  The JNI ABI is `int`, but we bind in C# as `uint`.

~~ ApiXmlAdjuster ~~

Update `JavaTypeReference` to contain unsigned types:

	UInt   = new JavaTypeReference ("uint");
	UShort = new JavaTypeReference ("ushort");
	ULong  = new JavaTypeReference ("ulong");
	UByte  = new JavaTypeReference ("ubyte");

~~ generator ~~

`generator` has the 4 new types added to the `SymbolTable` as
`SimpleSymbols`:

	AddType (new SimpleSymbol ("0", "uint",    "uint",   "I", returnCast: "(uint)"));
	AddType (new SimpleSymbol ("0", "ushort",  "ushort", "S", returnCast: "(ushort)"));
	AddType (new SimpleSymbol ("0", "ulong",   "ulong",  "J", returnCast: "(ulong)"));
	AddType (new SimpleSymbol ("0", "ubyte",   "byte",   "B", returnCast: "(byte)"));

There are 2 fixups we have to make because we use `GetIntValue(...)`,
etc. instead of having unsigned versions:

  * Override name of which method to call, e.g.: `GetIntValue()`
    instead of `GetUintValue()`.
  * Cast the `int` value returned to `uint`.  This is accomplished
    via the new `ISymbol.ReturnCast` property.

~~ A Note On API Compatibility ~~

Bindings which use Kotlin Unsigned Types will *only* work on
Xamarin.Android 10.2.0 or later ("Visual Studio 16.5").

The problem is that while we *can* emit C# source code which will
*compile* against older versions of Xamarin.Android, if they use
arrays they will not *run* under older versions of Xamarin.Android.

For example, imagine this binding code for the above Kotlin
`Example.array()` method:

	// C# Binding of Example.array()
	partial class Example {
	  public unsafe uint[] Array(uint[] value)
	  {
	    const string      __id          = "array--ajY-9A.([I)[I";
	    IntPtr            native_value  = JNIEnv.NewArray ((int[]) (object) value); // Works!...ish?
	    JniArgumentValue* __args        = stackalloc JniArgumentValue [1];
	    __args [0]                      = new JniArgumentValue (native_value);

	    JniObjectReference r            = _members.InstanceMethods.InvokeVirtualIntMethod (__id, this, __args);
	    return (uint[]) JNIEnv.GetArray (r.Handle, JniHandleOwnership.DoNotTransfer, typeof (uint));
	  }
	}

That could conceivably *compile* against older Xamarin.Android
versions.  However, that cannot *run* against older Xamarin.Android
versions, as *eventually* `JNIEnv.GetArray()` will hit some
dictionaries to determine how to marshal `IntPtr` to a `uint[]`, at
which point things will fail because there is no such mapping *until*
Xamarin.Android 10.2.0.

We feel that a "hard" ABI requirement will have more "graceful"
failure conditions than a solution which doesn't add ABI requirements.
In this case, if you create a Kotlin library binding which exposes
unsigned types, attempting to build an app in Release configuration
against older Xamarin.Android versions will result in a linker error,
as the required `JNIEnv` methods will not be resolvable.

(cherry picked from commit 71afce5)
jonpryor pushed a commit to dotnet/java-interop that referenced this pull request Jan 7, 2020
@jpobst @jonpryor
[generator] Support Kotlin's unsigned types (#539) (#553)
c0cc770 
Fixes: #525

Context: dotnet/android#4054
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Another place where Kotlin makes use of "name mangling" -- see also
commit f3553f4 -- is in the use of unsigned types such as `UInt`.
At the JVM ABI level, Kotlin treats unsigned types as their signed
counterparts, e.g. `kotlin.UInt` is an `int` and `kotlin.UIntArray`
is an `int[]`:

	// Kotlin
	public class Example {
	  public fun value(value: UInt) : UInt {
	    return value
	  }
	  public fun array(value: UIntArray) : UIntArray {
	    return value
	  }
	}

	// `javap` output:
	public final class Example {
	  public final int value-WZ4Q5Ns(int);
	  public final int[] array--ajY-9A(int[]);
	}

Kotlin uses Java Annotations to determine whether a parameter or
return type is actually an unsigned type instead of a signed type.

Update `Xamarin.Android.Tools.Bytecode` and `generator` to bind e.g.:

  * `kotlin.UInt` as `System.UInt32`
  * `kotlin.UIntArray` as a `System.UInt32[]`

and likewise for the other unsigned types `ushort`, `ulong`, `ubyte`.

In order to do this, we pretend that they are native Java types and
just translate a few places where we need to tell Java the real type.

~~ Xamarin.Android.Tools.Bytecode / class-parse ~~

When we read the Kotlin metadata in the Java bytecode, if we come
across one of these types we store it within an additional
`FieldInfo.KotlinType` property that we can access later.  When we
are generating the XML we check this additional flag and if it's one
of our types we emit it instead of the native Java type.

For example:

	<method
	  abstract="false"
	  deprecated="not deprecated"
	  final="false"
	  name="unsignedAbstractMethod-WZ4Q5Ns"
	  native="false"
	  return="uint"
	  jni-return="I"
	  static="false"
	  synchronized="false"
	  visibility="public"
	  bridge="false"
	  synthetic="false"
	  jni-signature="(I)I">
	  <parameter
	    name="value"
	    type="uint"
	    jni-type="I" />
	</method>

Here we see that even though `@jni-return` is `I` -- meaning `int` --
the `@return` property is `uint`.  Likewise `parameter/@jni-type` and
`parameter/@type`.  The JNI ABI is `int`, but we bind in C# as `uint`.

~~ ApiXmlAdjuster ~~

Update `JavaTypeReference` to contain unsigned types:

	UInt   = new JavaTypeReference ("uint");
	UShort = new JavaTypeReference ("ushort");
	ULong  = new JavaTypeReference ("ulong");
	UByte  = new JavaTypeReference ("ubyte");

~~ generator ~~

`generator` has the 4 new types added to the `SymbolTable` as
`SimpleSymbols`:

	AddType (new SimpleSymbol ("0", "uint",    "uint",   "I", returnCast: "(uint)"));
	AddType (new SimpleSymbol ("0", "ushort",  "ushort", "S", returnCast: "(ushort)"));
	AddType (new SimpleSymbol ("0", "ulong",   "ulong",  "J", returnCast: "(ulong)"));
	AddType (new SimpleSymbol ("0", "ubyte",   "byte",   "B", returnCast: "(byte)"));

There are 2 fixups we have to make because we use `GetIntValue(...)`,
etc. instead of having unsigned versions:

  * Override name of which method to call, e.g.: `GetIntValue()`
    instead of `GetUintValue()`.
  * Cast the `int` value returned to `uint`.  This is accomplished
    via the new `ISymbol.ReturnCast` property.

~~ A Note On API Compatibility ~~

Bindings which use Kotlin Unsigned Types will *only* work on
Xamarin.Android 10.2.0 or later ("Visual Studio 16.5").

The problem is that while we *can* emit C# source code which will
*compile* against older versions of Xamarin.Android, if they use
arrays they will not *run* under older versions of Xamarin.Android.

For example, imagine this binding code for the above Kotlin
`Example.array()` method:

	// C# Binding of Example.array()
	partial class Example {
	  public unsafe uint[] Array(uint[] value)
	  {
	    const string      __id          = "array--ajY-9A.([I)[I";
	    IntPtr            native_value  = JNIEnv.NewArray ((int[]) (object) value); // Works!...ish?
	    JniArgumentValue* __args        = stackalloc JniArgumentValue [1];
	    __args [0]                      = new JniArgumentValue (native_value);

	    JniObjectReference r            = _members.InstanceMethods.InvokeVirtualIntMethod (__id, this, __args);
	    return (uint[]) JNIEnv.GetArray (r.Handle, JniHandleOwnership.DoNotTransfer, typeof (uint));
	  }
	}

That could conceivably *compile* against older Xamarin.Android
versions.  However, that cannot *run* against older Xamarin.Android
versions, as *eventually* `JNIEnv.GetArray()` will hit some
dictionaries to determine how to marshal `IntPtr` to a `uint[]`, at
which point things will fail because there is no such mapping *until*
Xamarin.Android 10.2.0.

We feel that a "hard" ABI requirement will have more "graceful"
failure conditions than a solution which doesn't add ABI requirements.
In this case, if you create a Kotlin library binding which exposes
unsigned types, attempting to build an app in Release configuration
against older Xamarin.Android versions will result in a linker error,
as the required `JNIEnv` methods will not be resolvable.

(cherry picked from commit 71afce5)
@jonpryor jonpryor merged commit aa5e597 into master Jan 8, 2020
jonpryor pushed a commit that referenced this pull request Jan 8, 2020
@jpobst @jonpryor
[JNIEnv] Add support for Kotlin's unsigned types. (#4054)
7be87cf 
Fixes: dotnet/java-interop#525
Fixes: dotnet/java-interop#543

Context: dotnet/java-interop@71afce5
Context: https://github.com/Kotlin/KEEP/blob/13b67668ccc5b4741ecc37d0dd050fd77227c035/proposals/unsigned-types.md
Context: https://kotlinlang.org/docs/reference/basic-types.html#unsigned-integers

Kotlin has experimental (!) support for unsigned types, providing the
following value types:

  * `kotlin.UByte`: marshals as a Java `byte`/JNI `B`;
    equivalent to C# `byte`
  * `kotlin.UInt`: marshals as an Java `int`/JNI `I`;
    equivalent to C# `uint`
  * `kotlin.ULong`: marshals as a Java `long`/JNI `J`;
    equivalent to C# `ulong`
  * `kotlin.UShort`: marshals as a Java `short`/JNI `S`;
    equivalent to C# `ushort`

Kotlin also provides arrays of these types:

  * `kotlin.UByteArray`: marshals as a Java `byte[]`/JNI `[B`;
    equivalent to C# `byte[]`,
  * `kotlin.UIntArray`: marshals as an Java `int[]`/JNI `[I`;
    equivalent to C# `uint[]`.
  * `kotlin.ULongArray`: marshals as a Java `long[]`/JNI `[J`;
    equivalent to C# `ulong[]`.
  * `kotlin.UShortArray`: marshals as a Java `short[]`/JNI `[S`;
    equivalent to C# `ushort[]`.

Kotlin methods which contain unsigned types are "mangled", containing
characters which cannot be part of a valid Java identifier.  As such,
they cannot be invoked from Java code.

They *can* be invoked via JNI.

To bind these, we have two options:

 1. We could ignore all members which use unsigned types, or
 2. We could present a "Java-esque" view of the methods.

(1) is potentially problematic, as it means that abstract classes and
interfaces which contain them could become unbindable, making life
more complicated than is necessarily ideal.

(2) is viable, but ugly.

Consider this Kotlin type:

	// Kotlin
	package example;
	public open class ExampleBase {
	  public fun foo(value : UInt) {}
	}

Run `javap` on the resulting output, and we observe:

	Compiled from "hello.kt"
	public class example.ExampleBase {
	  public final void foo-WZ4Q5Ns(int);
	  public example.ExampleBase();
	}

We could bind `example.ExampleBase.foo-WZ4Q5Ns(int)` in C# by
replacing invalid characters with `_`, e.g.:

	// C# Binding
	partial class ExampleBase : Java.Lang.Object {
	  [Register ("foo-WZ4Q5Ns", …)]
	  public void Foo_WZ4Q5Ns (int value) {…}
	}

...but that would be *really* ugly.

We could instead just take everything before the `-` as the method
name for the C# method name -- resulting in `ExampleBase.Foo()` --
which results in a friendlier name from C#, but opens us up to
"collisions" with method overloads:

	// Kotlin
	public open class ExampleBase2 {
	  public fun foo(value : Int) {}
	  public fun foo(value : UInt) {}
	}

The C# `ExampleBase` type can't bind *both* `ExampleBase2.foo()`
methods as `Foo(int)`.

The chosen solution is to "more natively" support Kotlin unsigned
types, which allows reasonable disambiguation:

	// C# binding
	partial class ExampleBase2 : Java.Lang.Object {
	  [Register ("foo", …)]
	  public void Foo (int value) {…}

	  [Register ("foo-WZ4Q5Ns", …)]
	  public void Foo (uint value) {…}
	}

Java.Interop added support for emitting C# binding code which supports
Kotlin unsigned types.

For `kotlin.UByte`, `kotlin.UInt`, `kotlin.ULong`, and `kotlin.UShort`,
no additional runtime changes are needed to support the binding.

The `*Array` counterparts *do* require additional runtime changes.

Consider this Kotlin type:

	// Kotlin
	class UnsignedInstanceMethods {
	  public open fun uintArrayInstanceMethod (value: UIntArray) : UIntArray {
	    return value;
	  }
	}

The binding for this method would be:

	// C# binding
	partial class UnsignedInstanceMethods {
	  [Register ("uintArrayInstanceMethod--ajY-9A", "([I)[I", "")]
	  public unsafe uint[] UintArrayInstanceMethod (uint[] value)
	  {
	    const string __id = "uintArrayInstanceMethod--ajY-9A.([I)[I";
	    IntPtr native_value = JNIEnv.NewArray ((int[])(object)value);
	    try {
	      JniArgumentValue* __args = stackalloc JniArgumentValue [1];
	      __args [0] = new JniArgumentValue (native_value);
	      var __rm = _members.InstanceMethods.InvokeNonvirtualObjectMethod (__id, this, __args);
	      return (uint[]) JNIEnv.GetArray (__rm.Handle, JniHandleOwnership.TransferLocalRef, typeof (uint));
	    } finally {
	      if (value != null) {
	        JNIEnv.DeleteLocalRef (native_value);
	      }
	    }
	  }
	}

The problem is the `JNIEnv.GetArray(…, typeof(uint))` invocation.
This eventually hits various dictionaries within `JNIEnv`, which
requires additional support so that `uint` can be marshaled properly.

Previous versions of Xamarin.Android do not contain support for
marshaling arrays of unsigned types.

As such, support for using Kotlin bindings which contain unsigned
types will require Xamarin.Android 10.2.0 and later.

~~ Installer ~~

Add `protobuf-net.dll` to our installers, as it is required to read
the metadata that Kotlin puts within `.class` files.

~~ Unit Tests ~~

On-device tests for calling Kotlin code that uses unsigned types have
been added.

As we don't currently have a way of acquiring the Kotlin compiler for
builds, the test `.jar` and Kotlin's
`org.jetbrains.kotlin.kotlin-stdlib.jar` are copied into the repo.
They are *not* redistributed.

The source of the test `.jar` is provided for future use.

~~ Warnings ~~

Kotlin unsigned types are still experimental.  If Kotlin changes how
they are represented in Java bytecode, bindings which use them will
break.

Methods which use unsigned types *cannot* be virtual, nor can they
be used in bound interfaces.  This is because Java source code is
currently used as an "intermediary" for Java <-> Managed transitions,
and Java cannot override Kotlin "mangled" methods.  If the virtual
method is declared on a class, it will be bound as a non-`virtual`
method.  If the method is on an interface, it will be bound, but the
interface will not actually be implementable; a
[XA4213 build-time error][0] will be generated.

[0]: dotnet/java-interop@3bf5333
@jpobst jpobst deleted the unsigned-types branch January 8, 2020 15:04
@github-actions github-actions bot locked and limited conversation to collaborators Jan 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jonathanpeppers jonathanpeppers jonathanpeppers left review comments

@dellis1972 dellis1972 Awaiting requested review from dellis1972

@jonpryor jonpryor Awaiting requested review from jonpryor

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Published: 10.2 (d16-5)
Development

Successfully merging this pull request may close these issues.
4 participants
@jpobst @jonpryor @jonathanpeppers @brendanzagaeski