Developers can use System.Text.Json to serialize type hierarchies securely

[eiriktsarpalis]

Background and Motivation

Serializing type hierarchies (aka polymorphic serialization) has been a feature long requested by the community (cf. #29937, #30083). We took a stab at producing a polymorphism feature during the .NET 6 development cycle, ultimately though the feature was cut since we eventually reached the conclusion that unconstrained polymorphism does not meet our security bar, under any circumstance.

For context, polymorphic serialization has long been associated with security vulnerabilities. More specifically, unconstrained polymorphic serialization can result in accidental data disclosure and unconstrained polymorphic deserialization can result in remote code execution when used with untrusted input. See the BinaryFormatter security guide for an explanation of such vulnerabilities.

We acknowledge that serializing type hierarchies is an important feature, and we are committed to delivering a secure implementation in a future release of System.Text.Json. This implies that we will be releasing a brand of polymorphism that is restricted by design, requiring users to explicitly opt-in to supported subtypes.

Basic Polymorphism

At the core of the design is the introduction of the JsonDerivedType attribute:

[JsonDerivedType(typeof(Derived))]
public class Base
{
    public int X { get; set; }
}

public class Derived : Base
{
    public int Y { get; set; }
}

This configuration enables polymorphic serialization for Base, specifically when the runtime type is Derived:

Base value = new Derived();
JsonSerializer.Serialize<Base>(value); // { "X" : 0, "Y" : 0 }

Note that this does not enable polymorphic deserialization since the payload would roundtripped as Base:

Base value = JsonSerializer.Deserialize<Base>(@"{ ""X"" : 0, ""Y"" : 0 }");
value is Derived; // false

Polymorphism using Type Discriminators

To enable polymorphic deserialization, users need to specify a type discriminator for the derived class:

[JsonDerivedType(typeof(Base), typeDiscriminatorId: "base")]
[JsonDerivedType(typeof(Derived), typeDiscriminatorId: "derived")]
public class Base
{
    public int X { get; set; }
}

public class Derived : Base
{
    public int Y { get; set; }
}

Which will now emit JSON along with type discriminator metadata:

Base value = new Derived();
JsonSerializer.Serialize<Base>(value); // { "$type" : "derived", "X" : 0, "Y" : 0 }

which can be used to deserialize the value polymorphically:

Base value = JsonSerializer.Deserialize<Base>(@"{  ""$type"" : ""derived"", ""X"" : 0, ""Y"" : 0 }");
value is Derived; // true

Type discriminator identifiers can also be integers, so the following form is valid:

[JsonDerivedType(typeof(Derived1), 0)]
[JsonDerivedType(typeof(Derived2), 1)]
[JsonDerivedType(typeof(Derived3), 2)]
public class Base { }

JsonSerializer.Serialize<Base>(new Derived2()); // { "$type" : 1, ... }

Mixing and matching configuration

It is possible to mix and match type discriminator configuration for a given type hierarchy

[JsonPolymorphic]
[JsonDerivedType(typeof(Derived1))]
[JsonDerivedType(typeof(Derived2), "derived2")]
[JsonDerivedType(typeof(Derived3), 3)]
public class Base
{
    public int X { get; set; }
}

resulting in the following serializations:

var json1 = JsonSerializer.Serialize<Base>(new Derived1()); // { "X" : 0, "Y" : 0 }
var json2 = JsonSerializer.Serialize<Base>(new Derived2()); // { "$type" : "derived2", "X" : 0, "Z" : 0 }

Customizing Type Discriminators

It is possible to customize the property name of the type discriminator metadata like so:

[JsonPolymorphic(CustomTypeDiscriminatorPropertyName = "$case")]
[JsonDerivedType(typeof(Derived1), "derived1")]
public class Base
{
    public int X { get; set; }
}

resulting in the following JSON:

JsonSerializer.Serialize<Base>(new Derived1()); // { "$case" : "derived1", "X" : 0, "Y" : 0 }

Unknown Derived Type Handling

Consider the following type hierarchy:

[JsonDerivedType(typeof(DerivedType1))]
public class Base { }
public class DerivedType1 : Base { }
public class DerivedType2 : Base { }

Since the configuration does not explicitly opt-in support for DerivedType2, attempting to serialize instances of DerivedType2 as Base will result in a runtime exception:

JsonSerializer.Serialize<Base>(new DerivedType2()); // throws NotSupportedException

The default behavior can be tweaked using the JsonUnknownDerivedTypeHandling enum, which can be specified like so:

[JsonPolymorphic(UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FallBackToBaseType)]
[JsonDerivedType(typeof(DerivedType1))]
public class Base { }

JsonSerializer.Serialize<Base>(new DerivedType2()); // serialize using the contract for `Base`

The FallBackToNearestAncestor setting can be used to fall back to the contract of the nearest declared derived type:

[JsonPolymorphic(UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FallBackToNearestAncestor)]
[JsonDerivedType(typeof(MyDerivedClass)]
public interface IMyInterface { }
public class MyDerivedClass : IMyInterface { }

public class TestClass : MyDerivedClass { }

JsonSerializer.Serialize<IMyInterface>(new TestClass()); // serializes using the contract for `MyDerivedClass`

It should be noted that falling back to the nearest ancestor admits the possibility of diamond ambiguity:

[JsonPolymorphic(UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FallBackToNearestAncestor)]
[JsonDerivedType(typeof(MyDerivedClass)]
public interface IMyInterface { }

public interface IMyDerivedInterface : IMyInterface { }
public class MyDerivedClass : IMyInterface { }

public class TestClass : MyDerivedClass, IMyDerivedInterface { }

JsonSerializer.Serialize<IMyInterface>(new TestClass()); // throws NotSupportedException

Configuring Polymorphism via the Contract model

For use cases where attribute annotations are impractical or impossible (large domain models, cross-assembly hierarchies, hierarchies in third-party dependencies, etc.), it should still be possible to configure polymorphism using the JSON contract model:

public class MyPolymorphicTypeResolver : DefaultJsonTypeInfoResolver
{
    public override JsonTypeInfo GetTypeInfo(Type type, JsonSerializerOptions options)
    {
        JsonTypeInfo jsonTypeInfo = base.GetTypeInfo(type, options);
        if (jsonTypeInfo.Type == typeof(Base))
        {
            jsonTypeInfo.PolymorphismOptions =
                new JsonPolymorphismOptions
                {
                     TypeDiscriminatorPropertyName = "_case",
                     UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FallBackToNearestAncestor,
                     DerivedTypes =
                     {
                          new JsonDerivedType(typeof(DerivedType1)),
                          new JsonDerivedType(typeof(DerivedType2), "derivedType2"),
                          new JsonDerivedType(typeof(DerivedType3), 42),
                     }
               }
        }

        return jsonTypeInfo;
    }
}

Additional details

• Polymorphic serialization only supports derived types that have been explicitly opted in via the JsonDerivedType attribute. Undeclared runtime types will result in a runtime exception. The behavior can be changed by configuring the JsonPolymorphicAttribute.UnknownDerivedTypeHandling property.
• Polymorphic configuration specified in derived types is not inherited by polymorphic configuration in base types. These need to be configured independently.
• Polymorphic hierarchies are supported for both classes and interface types.
• Polymorphism using type discriminators is only supported for type hierarchies that use the default converters for objects, collections and dictionary types.
• Polymorphism is supported in metadata-based sourcegen, but not fast-path sourcegen.

API Proposal

namespace System.Text.Json.Serialization
{
    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Interface, AllowMultiple = false, Inherited = false)]
    public sealed class JsonPolymorphicAttribute : JsonAttribute
    {
        public string? TypeDiscriminatorPropertyName { get; set; }
        public bool IgnoreUnrecognizedTypeDiscriminators { get; set; }
        public JsonUnknownDerivedTypeHandling UnknownDerivedTypeHandling { get; set; }
    }

    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Interface, AllowMultiple = true, Inherited = false)]
    public class JsonDerivedTypeAttribute : JsonAttribute
    {
        public JsonDerivedTypeAttribute(Type derivedType);
        public JsonDerivedTypeAttribute(Type derivedType, string typeDiscriminatorId);
        public JsonDerivedTypeAttribute(Type derivedType, int typeDiscriminatorId);

        public Type DerivedType { get; }
        public object? TypeDiscriminatorId { get; }
    }

    public enum JsonUnknownDerivedTypeHandling
    {
        FailSerialization = 0, // Default, fail serialization on undeclared derived type
        FallBackToBaseType = 1, // Fall back to the base type contract
        FallBackToNearestAncestor = 2 // Fall back to the nearest declared derived type contract (admits diamond ambiguities in cases of interface hierarchies)
    }
}

namespace System.Text.Json.Metadata;

public partial class JsonTypeInfo
{
    public JsonPolymorphismOptions? PolymorphismOptions { get; set; } = null;
}

public class JsonPolymorphismOptions
{
    public JsonPolymorphismOptions();

    public ICollection<JsonDerivedType> DerivedTypes { get; }

    public bool IgnoreUnrecognizedTypeDiscriminators { get; set; } = false;
    public JsonUnknownDerivedTypeHandling UnknownDerivedTypeHandling { get; set; } = JsonUnknownDerivedTypeHandling.Default;
    public string TypeDiscriminatorPropertyName { get; set; } = "$type";
}

// Use a dedicated struct instead of ValueTuple that handles type checking of the discriminator id
public struct JsonDerivedType
{
    public JsonDerivedType(Type derivedType);
    public JsonDerivedType(Type derivedType, int typeDiscriminatorId);
    public JsonDerivedType(Type derivedType, string typeDiscriminatorId);

    public Type DerivedType { get; }
    public object? TypeDiscriminatorId { get; }
}

Anti-Goals

• No support for serializing open hierarchies (as specified in Support polymorphic serialization through new option #29937).
• No support for deserializing open hierarchies.

Progress

• Bring prototype branch up to date.
• Implementation & testing.
• API proposal & review.
• Conceptual documentation & blog posts.

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and Motivation

Serializing type hierarchies (aka polymorphic serialization) has been a long requested feature by the community (cf. #29937, #30083). We took a stab at producing a polymorphism feature during the .NET 6 development cycle, ultimately though the feature was cut since we eventually reached the conclusion that unconstrained polymorphism does not meet our security bar, under any circumstance.

For context, polymorphic serialization has long been associated with security vulnerabilities. More specifically, unconstrained polymorphic serialization can result in accidental data disclosure and unconstrained polymorphic deserialization can result in remote code execution when used with untrusted input. See the BinaryFormatter security guide for an explanation of such vulnerabilities.

We acknowledge that serializing type hierarchies is an important feature, and we are committed at delivering a secure implementation in a future release of System.Text.Json. This implies that we will be releasing a brand of polymorphism that is restricted by design, requiring users to explicitly opt-in to supported subtypes.

Examples

At the core of the design is the introduction of JsonKnownType attribute that can be applied to type hierarchies like so:

[JsonTypeDiscriminator("$type")]
[JsonKnownType(typeof(Derived1), "derived1")]
[JsonKnownType(typeof(Derived2), "derived2")]
public class Base
{
    public int X { get; set; }
}

public class Derived1 : Base
{
    public int Y { get; set; }
}

public class Derived2 : Base
{
    public int Z { get; set; }
}

This allows roundtrippable polymorphic serialization using the following schema:

var json1 = JsonSerializer.Serialize<Base>(new Derived1()); // { "$type" : "derived1", "X" : 0, "Y" : 0 }
var json2 = JsonSerializer.Serialize<Base>(new Derived2()); // { "$type" : "derived2", "X" : 0, "Z" : 0 }

JsonSerializer.Deserialize<Base>(json1) is Derived1; // true
JsonSerializer.Deserialize<Base>(json2) is Derived2; // true

Type discriminators can optionally be omitted from the contract like so:

[JsonKnownType(typeof(Derived1))]
[JsonKnownType(typeof(Derived2))]
public class Base
{
    public int X { get; set; }
}

resulting in the following serializations:

var json1 = JsonSerializer.Serialize<Base>(new Derived1()); // { "X" : 0, "Y" : 0 }
var json2 = JsonSerializer.Serialize<Base>(new Derived2()); // { "X" : 0, "Z" : 0 }

Note however that the particular contract does not support deserialization.

See also #30083 (comment) for details on the most recent API proposal.

Goals

• Users should be able to serialize and deserialize type hierarchies using type discriminators.
• Users must explicitly specify all supported subtypes for a given base type, following the approach used by DataContractSerializer's KnownTypeAttribute.
• Users can optionally omit type discriminators from the serialized value, losing the ability to roundtrip the serialized value.
• Users should be able to specify type hierarchies for types via APIs in JsonSerializerOptions.
• Users should be able to specify type hierarchies via customized JSON contracts.
• Type discriminators can either be of type string or type int, but a single hierarchy cannot mix and match types.
• We should be able to reuse the polymorphism infrastructure when delivering related features, such as System.Text.Json : Consider supporting F# discriminated unions #55744.

Anti-Goals

• No support for serializing open hierarchies (as specified in Support polymorphic serialization through new option #29937).
• No support for deserializing open hierarchies.

Open Questions

• Should we support polymorphic serialization for collection types?
• How should we handle serialization of types in the hierarchy not explicitly opted in (adopt contract of nearest ancestor vs. throwing an exception). The former approach suffers from diamond ambiguity in the context of interface hierarchies.
• How should we handle serialization of values of the base type if the base type has not been explicitly opted in?

Progress

• Bring prototype branch up to date.
• API proposal & review.
• Implementation & testing.
• Conceptual documentation & blog posts.

Author:	eiriktsarpalis
Assignees:	eiriktsarpalis
Labels:	area-System.Text.Json, User Story, Priority:2, Cost:L, Team:Libraries
Milestone:	7.0.0

[SCLDGit]

I'm curious for more explanation by the open question "Should we support polymorphic serialization for collection types?".

If it simply means "Should we allow users to serialize a collection of objects of various types that all derive from IType?" (e.g. List) , then I can't possibly imagine a scenario where the answer would be no.

[eiriktsarpalis]

If it simply means "Should we allow users to serialize a collection of objects of various types that all derive from IType?" (e.g. List) , then I can't possibly imagine a scenario where the answer would be no.

That's right. The issue primarily is that collections serialize as JSON arrays. As such introducing a type discriminator would serve to complicate the contract, i.e. it would likely require nesting the array payload inside an object with metadata:

{ "$type" : "derivedList", "$values" : [1,2,3,4,5] }

There is precedent for such encodings in the reference preservation feature.

[IFYates]

unconstrained polymorphic serialization can result in accidental data disclosure
(Anti-Goal) No support for serializing open hierarchies

I completely understand these arguments and decisions, but I think it's a shame that this option is taken away from the developer. An opt-in to serializing open hierarchies would have quickly bridged a large gap from Newtonsoft.Json (from what I've seen in other comments) and could prove useful in some diagnostic situations.
I've taken someone's example for a write-only polymorphic converter, which fills my needs but feels overdue for official support.

For the above examples, would it be too much to also support an inverse of the attribute:

[JsonTypeDiscriminator("$type")]
public class Base
{
    public int X { get; set; }
}

[JsonKnownDerivedType("derived2")]
public class Derived1 : Base
{
    public int Y { get; set; }
}

This provides a clean solution for when you don't own the Base class.
I feel like it might complicate deserialization, but even for a serialize-only approach, it feels better than having to maintain a central list of type mappings. (Assuming I've understood "Users should be able to specify type hierarchies for types via APIs in JsonSerializerOptions" correctly)

[SCLDGit]

It does feel a bit insulting to have a much-requested feature completely disallowed simply because people might misuse it. Even if we had to toss it in an unsafe block, it would be nice to have simple does-what-it-says-on-the-box JSON serialization/deserialization for the (very common) case where security is not a concern. I've used JSON as a quick, simple data interchange format (most recently to store hierarchical scene data for rendering software) more times than I can count, and not having to rely on Newtonsoft.JSON would be great for those cases.