JsonSerializer: Allow out-of-order reading of metadata properties.

[Hawxy]

EDIT See #72604 (comment) for an API proposal.

Description

Attempting to deserialize a polymorphic structure with System.Text.Json that doesn't feature $type at start of the object results in an exception.

For my scenario, this currently prevents me from using the built-in polymorphism support with jsonb columns in Postgres, as object properties have no guaranteed order.

Developer comment on this issue: #63747 (comment)

Reproduction Steps

var databaseState = @"{
  ""BaseDictionary"": {
    ""6ed6e524-2ca4-4fea-8e21-7245ccb61863"": {
      ""Id"": ""6ed6e524-2ca4-4fea-8e21-7245ccb61863"",
      ""Name"": ""Something"",
      ""$type"": ""Derived"",
      ""OtherGuid"": ""d471c77d-5412-4e7a-a98d-8304e87792ed""
    }
  }
}";

JsonSerializer.Deserialize<WrapperType>(databaseState);

public record WrapperType(Dictionary<Guid, WrapperType.Base> BaseDictionary)
{
    [JsonDerivedType(typeof(Derived), nameof(Derived))]
    [JsonDerivedType(typeof(AlsoDerived), nameof(AlsoDerived))]
    public abstract record Base(Guid Id, string Name);
    public record Derived(Guid Id, string Name, Guid OtherGuid): Base(Id, Name);
    public record AlsoDerived(Guid Id, string Name) : Base(Id, Name);
}

Expected behavior

Deserialization should work.

Actual behavior

System.NotSupportedException: 'Deserialization of types without a parameterless constructor, a singular parameterized constructor, or a parameterized constructor annotated with 'JsonConstructorAttribute' is not supported. 

Regression?

Limitation of initial implementation

Known Workarounds

I currently use PolyJson as an alternative, as the implementation reads ahead to find the discriminator.

Configuration

Impacts any version of STJ 7.0

Other information

No response

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Attempting to deserialize a polymorphic structure with System.Text.Json that doesn't feature $type at start of the object results in an exception.

For my scenario, this currently prevents me from using the built-in polymorphism support with jsonb columns in Postgres, as object properties have no guaranteed order.

Developer comment on this issue: #63747 (comment)

Reproduction Steps

var databaseState = @"{
  ""BaseDictionary"": {
    ""6ed6e524-2ca4-4fea-8e21-7245ccb61863"": {
      ""Id"": ""6ed6e524-2ca4-4fea-8e21-7245ccb61863"",
      ""Name"": ""Something"",
      ""$type"": ""Derived"",
      ""OtherGuid"": ""d471c77d-5412-4e7a-a98d-8304e87792ed""
    }
  }
}";

JsonSerializer.Deserialize<WrapperType>(databaseState);

public record WrapperType(Dictionary<Guid, WrapperType.Base> BaseDictionary)
{
    [JsonDerivedType(typeof(Derived), nameof(Derived))]
    [JsonDerivedType(typeof(AlsoDerived), nameof(AlsoDerived))]
    public abstract record Base(Guid Id, string Name);
    public record Derived(Guid Id, string Name, Guid OtherGuid): Base(Id, Name);
    public record AlsoDerived(Guid Id, string Name) : Base(Id, Name);
}

Expected behavior

Deserialization should work.

Actual behavior

System.NotSupportedException: 'Deserialization of types without a parameterless constructor, a singular parameterized constructor, or a parameterized constructor annotated with 'JsonConstructorAttribute' is not supported. 

Regression?

No response

Known Workarounds

I currently use PolyJson as an alternative, as the implementation reads ahead to find the discriminator.

Configuration

Impacts any version of STJ 7.0

Other information

No response

Author:	Hawxy
Assignees:	-
Labels:	area-System.Text.Json
Milestone:	-

[eiriktsarpalis]

Per #63747 (comment) this is a known limitation of the metadata reader. In the future we might consider adding read-ahead support specifically for type discriminators (I don't believe reference preservation to have such requirements), but this would be at the expense of potentially buffering excessive data in the case of async serialization.

[ilya-scale]

Is there any known workaround for this? I want to deserialize an object that does not come from System.Text.Json but has a discriminator that can be used that is of course not guaranteed to be first. The only thing I managed so far is to just deserialized first that one field and then specify type explicitly (i.e. not using the new polymorphic deserialization at all).

Maybe there is something else that can be done, like e.g. explicit reordering or something like that?

[eiriktsarpalis]

No workaround that doesn't involve writing a custom converter, unfortunately. In the future we might consider exposing a flag that reads ahead for type discriminators, but obviously that would come at the expense of performance so it would be turned off by default.

[ilya-scale]

Thanks for the answer! I assume the "future" will not be in .Net 7, so not coming anytime soon, right?

[eiriktsarpalis]

Correct, .NET 7 is currently in RC so feature development has been concluded.

[JobaDiniz]

Why can't I use my own property for the type discriminator?

My object has a string Type property already. According to this documentation
it says:

Avoid a JsonPolymorphicAttribute.TypeDiscriminatorPropertyName if it conflicts with a property in your type hierarchy.

I would like to tell System.Text.Json that there is a property (the first one) named Type, just use it.

[JsonPolymorphic(TypeDiscriminatorPropertyName = "type")]
    [JsonDerivedType(typeof(ChatbotSettings), typeDiscriminator: ChatbotSettings.TypeName)]
    [JsonDerivedType(typeof(UnattendedSettings), typeDiscriminator: UnattendedSettings.TypeName)]
    [JsonDerivedType(typeof(AttendedSettings), typeDiscriminator: AttendedSettings.TypeName)]
    internal abstract class RobotSettings
    {
        /// <summary>
        /// Do not use. This is for Json serialization.
        /// </summary>
        internal RobotSettings() { }

        internal RobotSettings(string type) => Type = type;

        internal required string Type { get; init; }
    }

[eiriktsarpalis]

Why can't I use my own property for the type discriminator?

The value of a string property can be arbitrary and might not necessarily reflect the runtime type of the object you are trying to serialize. In your example I would simply annotate the Type property with JsonIgnoreAttribute.

[kOchirasu]

I have this issue as well. When serializing JSON to MySQL using ef core, it seems that the ordering is not preserved so the data cannot be deserialized anymore.

There is a bit of a hack to work around this since MySQL seems to do deterministic ordering:

[JsonPolymorphic(TypeDiscriminatorPropertyName = "!")]

You might be able to get away with " " (space) as well, but basically a very low value ascii character that's valid will cause it to be sorted first. Definitely would not recommend doing this in production, but I am using this for myself.