System.Text.Json custom converter for Dictionary<string, object> collides with JsonExtensionData

[florianbader]

Description

Implementing a custom converter for a Dictionary<string, object> is used to serialize JsonExtensionData properties but there is no way for the converter to know if it is serializing an extension property or a dictionary property. Because of that invalid JSON is generated.

Reproduction Steps

See fiddle: https://dotnetfiddle.net/VIrtyu

Expected behavior

The custom converter should have a way to check if the current write operation value is json extension data.
{"RegularData":{"World":"Hello"},"Hello":"World"}

Actual behavior

The custom converter does not know if it json extension data and therefore writes an object instead of weaving the properties into the parent.
{"RegularData":{"World":"Hello"},{"Hello":"World"}}

Regression?

No response

Known Workarounds

No response

Configuration

.NET 5, System.Text.Json 5.0.2

Other information

Might be related to #32903

[jeffhandley]

Tagging @dotnet/area-system-text-json since the bot didn't.

[eiriktsarpalis]

Can reproduce, minimal repro:

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;

var value = new MyClass { ExtData = { ["Hello"] = "World" } };
var settings = new JsonSerializerOptions { Converters = { new CustomDictionaryConverter() } };
string json = JsonSerializer.Serialize(value, settings);
Console.WriteLine(json); // {42}

public class MyClass
{
    [JsonExtensionData]
    public Dictionary<string, object> ExtData { get; set; } = new Dictionary<string, object>();
}

public class CustomDictionaryConverter : JsonConverter<Dictionary<string, object?>>
{
    public override Dictionary<string, object?> Read(ref Utf8JsonReader reader, Type? typeToConvert, JsonSerializerOptions options)
        => throw new NotSupportedException();

    public override void Write(Utf8JsonWriter writer, Dictionary<string, object?> values, JsonSerializerOptions options)
        => writer.WriteNumberValue(42);
}

This is a bug, I think we need to make JsonExtensionData properties ignore custom converters, since it is bound to generate invalid JSON.

[nextdracool]

As shown in #60806 this also happens when using JsonObject. As a workaround you can define a converter, that writes the JsonObject's properties directly without writing the start and end of an object. You can then successfully serialize the object, but deserialization causes an exception: System.InvalidOperationException: 'A custom converter for JsonObject is not allowed on an extension property.'.

Given that it is apparently not allowed to add a custom converter to the JsonObject extension property during deserialization it would make sense to ignore present custom converters during serialization as well. Any explicitly added custom converter should probably throw the same exception when serializing as it does when deserializng.

[eiriktsarpalis]

Minimal reproduction using JsonObject without a custom converter:

using System;
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.Json.Serialization;

Person value = new() { Extensions = new() { ["Key"] = 42 } };
Console.WriteLine(JsonSerializer.Serialize(value));

public class Person
{
    [JsonExtensionData]
    public JsonObject? Extensions { get; set; }
}