Request: a tuple descriptor, so JSON messages can be smaller (currently requires InternalSerializationApi)

[aSemy]

What is your use-case and why do you need this feature?

I need to use JSON to encode and decode 2d arrays of x/y coordinates

  @Serializable
  data class Coordinates(
    val x: Int,
    val y: Int,
  )

By default, the JSON encoding is not size-optimal

{ "x": 1, "y": 2 }

Encoding a coordinate as a tuple (a fixed-length array, with fixed types per position, see the TypeScript docs) would make the messages more compact, and save a lot of space and improve processing speeds.

[1, 2]

There are other data structures I'd like to encode as well. For example, some information for each coordinate.

  @Serializable
  data class CoordinatesDetails(
    val x: Int,
    val y: Int,
    private val colour: String,
    private val active: Boolean,
  ) {
    ...
  }

Again, this would save a lot of space if I could en/decode it as a tuple

[1, 2, "red", true]

{ "x": 1, "y": 2, "colour": "red", "active": true }

Describe the solution you'd like

I've created a 'tuple descriptor builder' https://github.com/adamko-dev/kotlinx-serialization-typescript-generator/blob/main/docs/tuples.md

// tuple descriptor builder example
  @Serializable(with = Coordinates.Serializer::class)
  data class Coordinates(
    val x: Int,
    val y: Int,
  ) {
    object Serializer : TupleSerializer<Coordinates>(
      "Coordinates",
      {
        element(Coordinates::x)
        element(Coordinates::y)
      }
    ) {
      override fun tupleConstructor(elements: Iterator<*>): Coordinates {
        val x = requireNotNull(elements.next() as? Int)
        val y = requireNotNull(elements.next() as? Int)
        return Coordinates(x, y)
      }
    }
  }

This is quite finickity though, and also it's bugged. Under the hood it requires buildSerialDescriptor, because buildClassSerialDescriptor doesn't allow setting the structure kind to be LIST.

---

My custom tuple-serializer would be much easier if I could

1. retain the plugin-generated serializer Keep access to plugin generated serializer when mark class with @Serializable(with = MySerializer) #1169
2. override the serial kind of the plugin-generated descriptor and change it from CLASS to LIST.

---

Alternatively, it would be even easier if I could tell the compiler plugin to generate a descriptor as usual, except just change the 'kind' of the descriptor.

  @Serializable(kind = StructureKind.LIST)
  data class Coordinates(
    ...

---

Finally, another option is to add an arg to buildClassSerialDescriptor

public fun buildClassSerialDescriptor(
    serialName: String,
    serialKind: StructureKind = StructureKind.CLASS, // new arg, with a default 
    vararg typeParameters: SerialDescriptor,
    builderAction: ClassSerialDescriptorBuilder.() -> Unit = {}
):
...

[sandwwraith]

Yeah, this is indeed long-standing problem. I think we can pull buildSerialDescriptor out of internal API, since there is a demand for it.

[aSemy]

buildSerialDescriptor can work for my situation, but it's quite inconvenient and not very friendly to use. It requires that (at best) each 'tuple' class defines a custom serializer with manual casting. I imagine this will quite quickly get out of sync.

      override fun tupleConstructor(elements: Iterator<*>): Coordinates {
        val x = requireNotNull(elements.next() as? Int)
        val y = requireNotNull(elements.next() as? Int)
        return Coordinates(x, y)
      }

[joffrey-bion]

While answering this StackOverflow question, I realized it would indeed be useful to have something like a JsonTupleSerializer based on the JsonTransformingSerializer.

Here is an example of how it could look like:

/**
 * Serializes objects of type [T] into tuples of their properties, in the order defined by their serializer.
 */
open class JsonTupleSerializer<T : Any>(
    private val serializer: KSerializer<T>,
) : JsonTransformingSerializer<T>(serializer) {

    private val propertyNames = serializer.descriptor.elementNames.toList()

    override fun transformDeserialize(element: JsonElement): JsonElement =
        if (element is JsonArray) tupleToObject(element) else element

    private fun tupleToObject(element: JsonArray): JsonObject {
        require(element.size == propertyNames.size) {
            "Unexpected tuple length ${element.size} for ${serializer.descriptor.serialName}, " +
                "expected ${propertyNames.size} values for properties $propertyNames"
        }
        return buildJsonObject {
            propertyNames.forEachIndexed { index, propName ->
                put(propName, element[index])
            }
        }
    }

    override fun transformSerialize(element: JsonElement): JsonElement =
        if (element is JsonObject) objectToTuple(element) else element

    private fun objectToTuple(element: JsonObject) = buildJsonArray {
        propertyNames.forEach { propName ->
            // some properties might not be serialized in object form (e.g. default values)
            // but tuples are positional, so we need explicit nulls
            add(element[propName] ?: JsonNull)
        }
    }
}

[ephemient]

It might be slightly better to avoid putting null into an optional property when deserializing, in case it's a non-nullable property. I don't see a good way to handle optional nullable properties that are @EncodeDefault(NEVER) and have non-null defaults, though.

class JsonTupleSerializer<T : Any>(tSerializer: KSerializer<T>) : JsonTransformingSerializer<T>(tSerializer) {
    override val descriptor = SerialDescriptor(
        "JsonTupleSerializer<${tSerializer.descriptor.serialName}>",
        serialDescriptor<JsonArray>()
    )

    override fun transformDeserialize(element: JsonElement): JsonElement = if (element is JsonObject) buildJsonArray {
        for (name in super.descriptor.elementNames) add(element[name] ?: JsonNull)
    } else element

    override fun transformSerialize(element: JsonElement): JsonElement = if (element is JsonArray) buildJsonObject {
        val descriptor = super.descriptor
        for ((index, value) in element.jsonArray.withIndex()) {
            if (index >= descriptor.elementsCount) throw SerializationException("Too many values to unpack")
            if (value == JsonNull && descriptor.isElementOptional(index)) continue
            put(descriptor.getElementName(index), value)
        }
    } else element
}

[joffrey-bion]

Agreed. I overlooked this. I guess the idea of falling back to JsonNull to encode a missing property into the tuple was not great in general, not only for this special case. For non-nullable properties, it gives an escape hatch to represent a "non serialized default value", but for nullable properties this is incorrect, because null could mean 2 different things (actual null or non-serialized default).

I think we should not use JsonNull at all to represent defaults, and that gives us 2 options to deal with @EncodeDefault(NEVER):

1. forbid @EncodeDefault(NEVER) in JsonTupleSerializer entirely, on the grounds that it doesn't make sense for tuples because they are positional and need all values in their serialized form.
2. ignore @EncodeDefault(NEVER) and serialize the default value anyway (null or not), but if the default is not null, we might have to resort to reflection or more complex codegen to get the default value, because this transformation arrives "too late" - the JSON object element was already created without the property.

I would lean towards the first option, to be frank. Although this might lead to some inconvenience if NEVER is the default (and IIRC, it is). If we really can't take this inconvenience, we could decide to forbid @EncodeDefault(NEVER) only in the case where a property is nullable and has a non-null default value (and keep using nulls for missing values in other cases).

As a side note, is it on purpose that you swapped the serialization and deserialization? I think transformDeserialize is supposed to convert the serialized tuple (JsonArray) into an object (like in my initial proposal), not the other way around.

[ephemient]

Ah I didn't test it at all, just whipped up something in the IDE. I probably did get them reversed

I'm not sure if there's a way to force (effectively) @EncodeDefault(ALWAYS) in the presence of custom serializers which can do anything. Throw a runtime exception I suppose.