Support polymorphic serialization through new option

[steveharter]

EDIT see #29937 (comment) for the finalized API proposal.

Original proposal by @steveharter (click to view)

Current behavior

Consider a simple model:

abstract class Person {...}
class Customer : Person {...}

The current behavior, by design, is to use the static type and not serialize polymorphically.

// This will serialize all properties on `Person` but not `Customer`:
Person person = new Customer();
string str = JsonSerializer.ToString<Person>(person);

// This does the same thing through generic type inference:
string str = JsonSerializer.ToString(person);

However, if the type is declared to be System.Object then all properties will be serialized:

object person = new Customer();
// This will serialize all properties on `Customer`
string str = JsonSerializer.ToString(person);

In addition, the Type parameter can be specified:

Person person = new Customer();
// This will serialize all properties on `Customer`
string str = JsonSerializer.ToString(person, person.GetType());

All of these semantics are current as will remain as-is.

Proposed API

Add an opt-in setting that will serialize all types polymorphically, instead of just System.Object:

namespace System.Text.Json
{
    public class JsonSerializerOptions // existing class
    {
. . .
        bool SerializePolymorphically { get; set; }    
. . .
    }
}

Example of new behavior:

var options = new JsonSerializerOptions();
options.SerializePolymorphically = true;

// This will serialize all properties on `Customer`:
Person person = new Customer();
string str = JsonSerializer.ToString(person, options);

This new behavior applies both to the root object being serialized (as shown above) plus any properties on POCOs including collections, such as List<Person>.

Note: this issue does not enable any additional support for polymorphic deserialize. Currently, a property or collection of type System.Object will result in a JsonElement being created. Any additional polymorphic deserialize support must be done with a custom converter.

[steveharter]

Moving to future for the following reasons:

• 3.0 schedule is feature-complete in a few days.
• This may not be the best programming model long-term. We should really add feature at the same time we add support for polymorphic deserialization since we may want to share common options or an enum.
• Since we don't support true polymorphic deserialization, polymorphic serialization is not a useful for any round-tripping scenarios.
• The new custom converter feature can be used to handle polymorphic serialization of properties, as well as polymorphic deserialization of properties or types.

[david-driscoll]

I just ran into this scenario when doing some testing with latest preview. Is this something that is documented elsewhere? If not this should be highlighted in the migration guide as a new "feature" for new applications switching to use only the new JSON serializer. Otherwise someone might not notice the subtle change in their applications for a while.

[Symbai]

@steveharter The new custom converter feature cannot be used to handle polymorphic (de)serialization in some cases as, unless I'm missing something, because all properties have to be deserialized by hand / code. Which is not an option for maintainability when you have classes with more than two properties. See https://github.com/dotnet/corefx/issues/39031

The issue is that in the custom converter you only have a Utf8JsonReader which can read but not deserialize. Means you have to read one-by-one and set properties one-by-one.

If there however would be a way to deserialize an object, but only the object not the whole json, within the custom converter all at once, it becomes a feature that can be used.

[steveharter]

@Symbai yes you are correct for deserialization -- there is not an easy way to support without manually authoring.

Here's a test\sample for using a type discriminator to do that:

Also here's a test\sample for polymorphic serialization. Currently the test\sample assumes the base class is abstract.

I'll work on adding real doc and samples shortly.

[Symbai]

@steveharter Thanks but the samples cannot be used in real world scenarios because classes mostly don't have only 2 properties as shown in the sample, but ten to fifty. And now imagine we serialize something that itself has a class as property and this class also has another class and so on. So in the end, we have over hundreds of properties we, currently, have to write for EACH of them this here:

 case "Name of property":
                                ... bla= reader.GetXY();
                                ((MyClass)value).Bla= bla;
                                break;

But if you have to write this more than 10 times already the code gets ugly and hardly maintainable.

Example

public override Person Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
	if (reader.TokenType != JsonTokenType.StartObject)
	{
		throw new JsonException();
	}

	reader.Read();
	if (reader.TokenType != JsonTokenType.PropertyName)
	{
		throw new JsonException();
	}

	string propertyName = reader.GetString();
	if (propertyName != "TypeDiscriminator")
	{
		throw new JsonException();
	}

	reader.Read();
	if (reader.TokenType != JsonTokenType.Number)
	{
		throw new JsonException();
	}

	Person value;
	TypeDiscriminator typeDiscriminator = (TypeDiscriminator)reader.GetInt32();
	switch (typeDiscriminator)
	{
		case TypeDiscriminator.Customer:
			value = new Customer();
			break;

		case TypeDiscriminator.Employee:
			value = new Employee();
			break;

		default:
			throw new JsonException();
	}

	while (reader.Read())
	{
		if (reader.TokenType == JsonTokenType.EndObject)
		{
			return value;
		}

		if (reader.TokenType == JsonTokenType.PropertyName)
		{
			propertyName = reader.GetString();
			reader.Read();
			switch (propertyName)
			{
				case "0":
					decimal zero = reader.GetDecimal();
					((Class1)value).ZERO = zero;
					break;
				case "1":
					decimal one = reader.GetDecimal();
					((Class1)value).ONE = one;
					break;
				case "2":
					decimal two = reader.GetDecimal();
					((Class1)value).TWO = two;
					break;
				case "3":
					decimal three = reader.GetDecimal();
					((Class1)value).THREE = three;
					break;
				case "4":
					decimal four = reader.GetDecimal();
					((Class1)value).FOUR = four;
					break;
				case "5":
					decimal five = reader.GetDecimal();
					((Class1)value).FIVE = five;
					break;
				case "6":
					decimal six = reader.GetDecimal();
					((Class1)value).SIX = six;
					break;
				case "7":
					decimal seven = reader.GetDecimal();
					((Class1)value).SEVEN = seven;
					break;
				case "8":
					decimal eight = reader.GetDecimal();
					((Class1)value).EIGHT = eight;
					break;
				case "9":
					decimal nine = reader.GetDecimal();
					((Class1)value).NINE = nine;
					break;
				case "10":
					decimal ten = reader.GetDecimal();
					((Class1)value).TEN = ten;
					break;
				case "11":
					decimal eleven = reader.GetDecimal();
					((Class2)value).ELEVEN = eleven;
					break;
				case "12":
					decimal twelve = reader.GetDecimal();
					((Class2)value).TWELVE = twelve;
					break;
				case "13":
					decimal thirteen = reader.GetDecimal();
					((Class2)value).THIRTEEN = thirteen;
					break;
				case "14":
					decimal fourteen = reader.GetDecimal();
					((Class2)value).FOURTEEN = fourteen;
					break;
				case "15":
					decimal fifteen = reader.GetDecimal();
					((Class2)value).FIFTEEN = fifteen;
					break;
				case "16":
					decimal sixteen = reader.GetDecimal();
					((Class2)value).SIXTEEN = sixteen;
					break;
				case "17":
					decimal seventeen = reader.GetDecimal();
					((Class2)value).SEVENTEEN = seventeen;
					break;
				case "18":
					decimal eighteen = reader.GetDecimal();
					((Class2)value).EIGHTEEN = eighteen;
					break;
				case "19":
					decimal nineteen = reader.GetDecimal();
					((Class2)value).NINETEEN = nineteen;
					break;
				case "20":
					decimal twenty = reader.GetDecimal();
					((Class2)value).TWENTY = twenty;
					break;
				case "21":
					decimal twenty-one = reader.GetDecimal();
					((Class3)value).TWENTY-ONE = twenty-one;
					break;
				case "22":
					decimal twenty-two = reader.GetDecimal();
					((Class3)value).TWENTY-TWO = twenty-two;
					break;
				case "23":
					decimal twenty-three = reader.GetDecimal();
					((Class3)value).TWENTY-THREE = twenty-three;
					break;
				case "24":
					decimal twenty-four = reader.GetDecimal();
					((Class3)value).TWENTY-FOUR = twenty-four;
					break;
				case "25":
					decimal twenty-five = reader.GetDecimal();
					((Class3)value).TWENTY-FIVE = twenty-five;
					break;
				case "26":
					decimal twenty-six = reader.GetDecimal();
					((Class3)value).TWENTY-SIX = twenty-six;
					break;
				case "27":
					decimal twenty-seven = reader.GetDecimal();
					((Class3)value).TWENTY-SEVEN = twenty-seven;
					break;
				case "28":
					decimal twenty-eight = reader.GetDecimal();
					((Class3)value).TWENTY-EIGHT = twenty-eight;
					break;
				case "29":
					decimal twenty-nine = reader.GetDecimal();
					((Class3)value).TWENTY-NINE = twenty-nine;
					break;
				case "30":
					decimal thirty = reader.GetDecimal();
					((Class3)value).THIRTY = thirty;
					break;
				case "31":
					decimal thirty-one = reader.GetDecimal();
					((Class4)value).THIRTY-ONE = thirty-one;
					break;
				case "32":
					decimal thirty-two = reader.GetDecimal();
					((Class4)value).THIRTY-TWO = thirty-two;
					break;
				case "33":
					decimal thirty-three = reader.GetDecimal();
					((Class4)value).THIRTY-THREE = thirty-three;
					break;
				case "34":
					decimal thirty-four = reader.GetDecimal();
					((Class4)value).THIRTY-FOUR = thirty-four;
					break;
				case "35":
					decimal thirty-five = reader.GetDecimal();
					((Class4)value).THIRTY-FIVE = thirty-five;
					break;
				case "36":
					decimal thirty-six = reader.GetDecimal();
					((Class4)value).THIRTY-SIX = thirty-six;
					break;
				case "37":
					decimal thirty-seven = reader.GetDecimal();
					((Class4)value).THIRTY-SEVEN = thirty-seven;
					break;
				case "38":
					decimal thirty-eight = reader.GetDecimal();
					((Class4)value).THIRTY-EIGHT = thirty-eight;
					break;
				case "39":
					decimal thirty-nine = reader.GetDecimal();
					((Class4)value).THIRTY-NINE = thirty-nine;
					break;
				case "40":
					decimal forty = reader.GetDecimal();
					((Class4)value).FORTY = forty;
					break;
				case "41":
					decimal forty-one = reader.GetDecimal();
					((Class5)value).FORTY-ONE = forty-one;
					break;
				case "42":
					decimal forty-two = reader.GetDecimal();
					((Class5)value).FORTY-TWO = forty-two;
					break;
				case "43":
					decimal forty-three = reader.GetDecimal();
					((Class5)value).FORTY-THREE = forty-three;
					break;
				case "44":
					decimal forty-four = reader.GetDecimal();
					((Class5)value).FORTY-FOUR = forty-four;
					break;
				case "45":
					decimal forty-five = reader.GetDecimal();
					((Class5)value).FORTY-FIVE = forty-five;
					break;
				case "46":
					decimal forty-six = reader.GetDecimal();
					((Class5)value).FORTY-SIX = forty-six;
					break;
				case "47":
					decimal forty-seven = reader.GetDecimal();
					((Class5)value).FORTY-SEVEN = forty-seven;
					break;
				case "48":
					decimal forty-eight = reader.GetDecimal();
					((Class5)value).FORTY-EIGHT = forty-eight;
					break;
				case "49":
					decimal forty-nine = reader.GetDecimal();
					((Class5)value).FORTY-NINE = forty-nine;
					break;
				case "50":
					decimal fifty = reader.GetDecimal();
					((Class5)value).FIFTY = fifty;
					break;
				case "51":
					decimal fifty-one = reader.GetDecimal();
					((Class6)value).FIFTY-ONE = fifty-one;
					break;
				case "52":
					decimal fifty-two = reader.GetDecimal();
					((Class6)value).FIFTY-TWO = fifty-two;
					break;
				case "53":
					decimal fifty-three = reader.GetDecimal();
					((Class6)value).FIFTY-THREE = fifty-three;
					break;
				case "54":
					decimal fifty-four = reader.GetDecimal();
					((Class6)value).FIFTY-FOUR = fifty-four;
					break;
				case "55":
					decimal fifty-five = reader.GetDecimal();
					((Class6)value).FIFTY-FIVE = fifty-five;
					break;
				case "56":
					decimal fifty-six = reader.GetDecimal();
					((Class6)value).FIFTY-SIX = fifty-six;
					break;
				case "57":
					decimal fifty-seven = reader.GetDecimal();
					((Class6)value).FIFTY-SEVEN = fifty-seven;
					break;
				case "58":
					decimal fifty-eight = reader.GetDecimal();
					((Class6)value).FIFTY-EIGHT = fifty-eight;
					break;
				case "59":
					decimal fifty-nine = reader.GetDecimal();
					((Class6)value).FIFTY-NINE = fifty-nine;
					break;
				case "60":
					decimal sixty = reader.GetDecimal();
					((Class6)value).SIXTY = sixty;
					break;
				case "61":
					decimal sixty-one = reader.GetDecimal();
					((Class7)value).SIXTY-ONE = sixty-one;
					break;
				case "62":
					decimal sixty-two = reader.GetDecimal();
					((Class7)value).SIXTY-TWO = sixty-two;
					break;
				case "63":
					decimal sixty-three = reader.GetDecimal();
					((Class7)value).SIXTY-THREE = sixty-three;
					break;
				case "64":
					decimal sixty-four = reader.GetDecimal();
					((Class7)value).SIXTY-FOUR = sixty-four;
					break;
				case "65":
					decimal sixty-five = reader.GetDecimal();
					((Class7)value).SIXTY-FIVE = sixty-five;
					break;
				case "66":
					decimal sixty-six = reader.GetDecimal();
					((Class7)value).SIXTY-SIX = sixty-six;
					break;
				case "67":
					decimal sixty-seven = reader.GetDecimal();
					((Class7)value).SIXTY-SEVEN = sixty-seven;
					break;
				case "68":
					decimal sixty-eight = reader.GetDecimal();
					((Class7)value).SIXTY-EIGHT = sixty-eight;
					break;
				case "69":
					decimal sixty-nine = reader.GetDecimal();
					((Class7)value).SIXTY-NINE = sixty-nine;
					break;
				case "70":
					decimal seventy = reader.GetDecimal();
					((Class7)value).SEVENTY = seventy;
					break;
				case "71":
					decimal seventy-one = reader.GetDecimal();
					((Class8)value).SEVENTY-ONE = seventy-one;
					break;
				case "72":
					decimal seventy-two = reader.GetDecimal();
					((Class8)value).SEVENTY-TWO = seventy-two;
					break;
				case "73":
					decimal seventy-three = reader.GetDecimal();
					((Class8)value).SEVENTY-THREE = seventy-three;
					break;
				case "74":
					decimal seventy-four = reader.GetDecimal();
					((Class8)value).SEVENTY-FOUR = seventy-four;
					break;
				case "75":
					decimal seventy-five = reader.GetDecimal();
					((Class8)value).SEVENTY-FIVE = seventy-five;
					break;
				case "76":
					decimal seventy-six = reader.GetDecimal();
					((Class8)value).SEVENTY-SIX = seventy-six;
					break;
				case "77":
					decimal seventy-seven = reader.GetDecimal();
					((Class8)value).SEVENTY-SEVEN = seventy-seven;
					break;
				case "78":
					decimal seventy-eight = reader.GetDecimal();
					((Class8)value).SEVENTY-EIGHT = seventy-eight;
					break;
				case "79":
					decimal seventy-nine = reader.GetDecimal();
					((Class8)value).SEVENTY-NINE = seventy-nine;
					break;
				case "80":
					decimal eighty = reader.GetDecimal();
					((Class8)value).EIGHTY = eighty;
					break;
				case "81":
					decimal eighty-one = reader.GetDecimal();
					((Class9)value).EIGHTY-ONE = eighty-one;
					break;
				case "82":
					decimal eighty-two = reader.GetDecimal();
					((Class9)value).EIGHTY-TWO = eighty-two;
					break;
				case "83":
					decimal eighty-three = reader.GetDecimal();
					((Class9)value).EIGHTY-THREE = eighty-three;
					break;
				case "84":
					decimal eighty-four = reader.GetDecimal();
					((Class9)value).EIGHTY-FOUR = eighty-four;
					break;
				case "85":
					decimal eighty-five = reader.GetDecimal();
					((Class9)value).EIGHTY-FIVE = eighty-five;
					break;
				case "86":
					decimal eighty-six = reader.GetDecimal();
					((Class9)value).EIGHTY-SIX = eighty-six;
					break;
				case "87":
					decimal eighty-seven = reader.GetDecimal();
					((Class9)value).EIGHTY-SEVEN = eighty-seven;
					break;
				case "88":
					decimal eighty-eight = reader.GetDecimal();
					((Class9)value).EIGHTY-EIGHT = eighty-eight;
					break;
				case "89":
					decimal eighty-nine = reader.GetDecimal();
					((Class9)value).EIGHTY-NINE = eighty-nine;
					break;
				case "90":
					decimal ninety = reader.GetDecimal();
					((Class9)value).NINETY = ninety;
					break;
				case "91":
					decimal ninety-one = reader.GetDecimal();
					((Class10)value).NINETY-ONE = ninety-one;
					break;
				case "92":
					decimal ninety-two = reader.GetDecimal();
					((Class10)value).NINETY-TWO = ninety-two;
					break;
				case "93":
					decimal ninety-three = reader.GetDecimal();
					((Class10)value).NINETY-THREE = ninety-three;
					break;
				case "94":
					decimal ninety-four = reader.GetDecimal();
					((Class10)value).NINETY-FOUR = ninety-four;
					break;
				case "95":
					decimal ninety-five = reader.GetDecimal();
					((Class10)value).NINETY-FIVE = ninety-five;
					break;
				case "96":
					decimal ninety-six = reader.GetDecimal();
					((Class10)value).NINETY-SIX = ninety-six;
					break;
				case "97":
					decimal ninety-seven = reader.GetDecimal();
					((Class10)value).NINETY-SEVEN = ninety-seven;
					break;
				case "98":
					decimal ninety-eight = reader.GetDecimal();
					((Class10)value).NINETY-EIGHT = ninety-eight;
					break;
				case "99":
					decimal ninety-nine = reader.GetDecimal();
					((Class10)value).NINETY-NINE = ninety-nine;
					break;

			}
		}
	}

	throw new JsonException();
}

[AlexanderJohnston]

This seriously needs to be documented somewhere, like on the System.Text.Json namespace, in highlighted text. I spent hours trying to track down why it was serializing empty strings while working in a Blazor project.

[knom]

This is bad.. just ran into this as well.. after hours of debugging :-(
Is there a workaround using Newtonsoft.JSON?

[steentottrup]

My own findings:

https://stackoverflow.com/questions/58305928/the-new-asp-net-core-3-0-json-serializer-is-leaving-out-data/58316304#58316304

And an article on how to "go back" to Newtonsoft.Json:

https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis/

[Symbai]

Is there a workaround using Newtonsoft.JSON?

On Newtonsoft it's super easy. For example just tell Newtonsoft all known Types and Newtonsoft does all the magic itself for you. No converter, no worries:

public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings {
	TypeNameHandling = TypeNameHandling.Objects,
	SerializationBinder = new KnownTypesBinder {
		KnownTypes = new List<Type> {
			typeof(TypeA),
			typeof(TypeB),
			typeof(TypeC),
			typeof(TypeD),
			typeof(TypeE)
		}
}};

I really wish this would get more attention, at least set to 5.0 milestone instead of future. Currently it's just unusable for polymorphic situations and there is NO workaround (I consider the converter as no workaround in real world scenarios for the reason I posted earlier), so switching is not possible.

[steveharter]

Moving to 5.0.

However this should also be tied to polymorphic deserialization support as well which is also a requested feature but much larger in scope. There are several requests for that and the implementation must be secure and opt-in through known types; we need to combine\triage these polymorphic issues -- here's one request: https://github.com/dotnet/corefx/issues/41347

[bklooste]

Known types / magic polymorphism on the wire are bad patterns and I have seen people waste a ton of time on weird issues and bugs around this - the benefit aint worth it. Neither Json or XML are polymorphic by design and making them do it is a hack at best.

That said we should document explicit type creation better and possibly have some patterns around this eg

var typeName = json.GetProperty("TypeName").GetString(); var type = Type.GetType(typeName); return (T) JsonSerializer.Deserialize(utfJson , type);