JsonSerializer.Deserialize fails to deserialize to {ReadOnly}Memory<T>

[rampaa]

Description

When trying to deserialize a JSON to {ReadOnly}Memory<T>, JsonSerializer.Deserialize fails for some perfectly valid JSON files.

The following code prints Deserialization failed miserably!:

        using FileStream fileStream = File.OpenRead("Names.json");
        ReadOnlyMemory<NameRecord> nameRecords = JsonSerializer.Deserialize<ReadOnlyMemory<NameRecord>>(fileStream);
        if (nameRecords.Length is 0)
        {
            Console.WriteLine("Deserialization failed miserably!");
        }
        else
        {
            Console.WriteLine("Deserialization succeeded!");
        }

While the following code prints Deserialization succeeded!:

        using FileStream fileStream = File.OpenRead("Names.json");
        ReadOnlyMemory<NameRecord> nameRecords = JsonSerializer.Deserialize<NameRecord[]>(fileStream);
        if (nameRecords.Length is 0)
        {
            Console.WriteLine("Deserialization failed miserably!");
        }
        else
        {
            Console.WriteLine("Deserialization succeeded!");
        }

Potentially related to #118346

---

Should we just not use {ReadOnly}Memory as a deserialization target? Is its usage some kind of an edge case that is not well tested? I was under the impression that if I don't strictly need a List<T> or T[], deserializing a JSON file to ReadOnlyMemory<T> would have some performance benefits (like allocating less) but it causing all these issues for random JSON files makes me wonder whether I am missing a memo where its usage is discouraged for JSON deserialization.

Reproduction Steps

1. Run ReadOnlyMemoryDeserializationBug.zip
2. Notice it prints Deserialization failed miserably!
3. Change JsonSerializer.Deserialize<ReadOnlyMemory<NameRecord>>(fileStream) to JsonSerializer.Deserialize<NameRecord[]>(fileStream) and re-run it.
4. Notice it prints Deserialization succeeded!

Expected behavior

JsonSerializer.Deserialize<ReadOnlyMemory<T>> should be able to deserialize everything JsonSerializer.Deserialize<T[]> can deserialize.

Actual behavior

JsonSerializer.Deserialize<ReadOnlyMemory<T>> fails to deserialize some JSON files that JsonSerializer.Deserialize<T[]> deserializes just fine.

Regression?

Partially. For some JSON files .NET 8 does not fail but .NET 9 fails. For some other JSON files .NET 8 fails with an (incorrect) exception but .NET 9 fails silently. The attached repro project showcases where both .NET versions fail.

Known Workarounds

Deserializing to List<T> or T[] works fine

Configuration

Which version of .NET is the code running on?

.NET 9

What OS and version, and what distro if applicable?

Windows 10 22H2

What is the architecture (x64, x86, ARM, ARM64)?

x64

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[huoyaoyuan]

With #86442, ReadOnlyMemory<T> is now treated as T[]. The following code works:

JsonSerializer.Deserialize<ReadOnlyMemory<int>>("[1, 2, 3, 4, 5, 6]");

Can you provide a full reproduction for the issue?

I was under the impression that if I don't strictly need a List<T> or T[], deserializing a JSON file to ReadOnlyMemory<T> would have some performance benefits (like allocating less)

This is never the case. The ReadOnlyMemory<T> converter just deserializes into array and wraps it to ReadOnlyMemory<T>. It doesn't do the optimization that slicing on larger buffer.

[rampaa]

With #86442, ReadOnlyMemory is now treated as T[].

It obviously does not act the same way for ReadOnlyMemory<T> and T[] though. Case-in-point, this issue itself. Another case-in-point: #118346 (comment)

The following code works:
JsonSerializer.Deserialize<ReadOnlyMemory<int>>("[1, 2, 3, 4, 5, 6]");

I am sure it does. I was not trying to say JsonSerializer.Deserialize never successfully deserialize to ReadOnlyMemory<T>. It just can't do it for some perfectly valid JSON files.

Can you provide a full reproduction for the issue?

Reproduction Steps

1. Run ReadOnlyMemoryDeserializationBug.zip
2. Notice it prints Deserialization failed miserably!
3. Change JsonSerializer.Deserialize<ReadOnlyMemory<NameRecord>>(fileStream) to JsonSerializer.Deserialize<NameRecord[]>(fileStream) and re-run it.
4. Notice it prints Deserialization succeeded!

This is never the case. The ReadOnlyMemory<T> converter just deserializes into array and wraps it to ReadOnlyMemory<T>. It doesn't do the optimization that slicing on larger buffer.

So deserializing to ReadOnlyMemory<T> instead of T[] is basically meaningless as far as performance is concerned, thanks for letting me know.

[eiriktsarpalis]

I can confirm that this is a bug in the built in Memory and ReadOnlyMemory converters when performing streaming serialization. The null check at the start of the OnTryRead method:

runtime/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/ReadOnlyMemoryConverter.cs

Lines 21 to 25 in 2e85cbd

	if (reader.TokenType is JsonTokenType.Null)
	{
	value = default;
	return true;
	}

Does not take resumption into account and returns early if the converter is attempting to resume when the current (nested) JSON token happens to be null. A fix should additionally include a state.IsContinuation check in each of the two converters.