Writing raw JSON values when using Utf8JsonWriter

[nahk-ivanov]

Edited by @layomia

Original proposal by @nahk-ivanov (click to view)

According to the image in the roadmap - Utf8JsonWriter would be the lowest level of API we are going to get, but it's missing WriteRawValue functionality from JSON.NET. Seems like it is "by design", as roadmap states

Does not allow writing invalid JSON according to the JSON RFC

In this case, it would be nice to have access to the output stream directly in the custom JsonConverter, as there are still scenarios where for performance we want to write raw JSON. It's quite often that application layer doesn't care about particular branches of the JSON document and just wants to pass them through to/from the data store (especially when using NoSQL). What we used to do with JSON.NET in such cases is maintain it as string property and define custom converter that would use WriteRawValue, which took a string argument.

One of the examples would be using Azure Table Storage, where we want to extract few top-level properties to be PartitionKey/RowKey and store other nested properties in their original JSON form. I don't see any reason why application would unnecessarily deserialize and serialize these parts of the tree - all we need is to just run forward-only reader to return the value of the given property in it's raw form and later write it as such.

I looked at the current System.Text.Json APIs and can't find a good way of doing this. It seems like (for performance reasons) we would want to maintain raw data in a binary form now (compared to string previously), which is fine, but there is no way to read/write next value as bytes. The best I could find is to use JsonDocument.ParseValue + document.RootElement.GetRawText() when reading it and parse it again into a JsonDocument when writing to just call WriteTo.

Am I missing some APIs?

---

Introduction

There are scenarios where customers want to integrate existing, "raw" JSON payloads when writing new JSON payloads with Utf8JsonWriter. There's no first-class API for providing this functionality today, but there's a workaround using JsonDocument:

// UTF8 representation of {"Hello":"World"}
byte[] rawJson = GetRawPayload();

writer.WriteStartObject();
writer.WritePropertyName("Payload");

// No easy way to write this raw JSON payload. Here's a workaround:
using (JsonDocument document = JsonDocument.Parse(rawJson))
{
    document.RootElement.WriteTo(writer);
}

writer.WriteEndObject():

This implementation builds a metadata database for navigating a JSON document, which is unnecessary given the desired functionality and has a performance overhead.

The goal of this feature is to provide performant and safe APIs to write raw JSON values.

API Proposal

namespace System.Text.Json
{
  public sealed partial class Utf8JsonWriter
  {
    // Writes the span of bytes directly as JSON content.
    public void WriteRawValue(ReadOnlySpan<byte> utf8Json, bool skipValidation = false) { }

    // Writes the span of bytes directly as JSON content.
    public void WriteRawValue(ReadOnlySpan<char> json, bool skipValidation = false) { }

    // Writes the span of bytes directly as JSON content.
    public void WriteRawValue(string json, bool skipValidation = false) { }
  }
}

Alternate Design (click to view)

namespace System.Text.Json
{
  // Specifies processing options when writing raw JSON values.
  [Flags]
  public enum JsonWriteRawOptions
  {
    // Raw JSON values will be validated for structural correctness and encoded if required.
    Default = 0,

    // Whether to skip validation. Independent of JsonWriterOptions.SkipValidation.
    SkipValidation = 1,

    // Whether to skip encoding specified via JsonWriterOptions.Encoder.
    SkipEncoding = 2,
  }

  public sealed partial class Utf8JsonWriter
  {
    // Writes the span of bytes directly as JSON content, according to the specified options.
    public void WriteRawValue(ReadOnlySpan<byte> utf8Json, JsonWriteRawOptions options = default) { }
  }
}

Potential additions

namespace System.Text.Json
{
  [Flags]
  public enum JsonWriteRawOptions
  {
    // Existing
    // Default = 0,
    // SkipValidation = 1,
    // SkipEncoding = 2,

    // Whether to reformat to raw payload according to the JsonWriterOptions, or write it as-is.
    // Reformatting is influenced by JsonWriterOptions.Indented, and may result in whitespace changes.
    Reformat = 4
  }

  public sealed partial class Utf8JsonWriter
  {
    // Existing
    // public void WriteRawValue(ReadOnlySpan<byte> utf8Json, JsonWriteRawOptions? options = null) { }

    // Write raw value overload that takes ROS<char>.
    public void WriteRawValue(ReadOnlySpan<char> json, JsonWriteRawOptions options = default) { }
    
    // Write raw value overload that takes string.
    public void WriteRawValue(string json, JsonWriteRawOptions options = default) { }
  }
}

Scenarios

The major considerations for this feature are:

• Should we validate raw payloads for structural correctness?
• Should we encode the raw JSON payload according to the writer options?
• Should we maintain the formatting of the raw payload when writing, or should we re-format it according to the writer options?

The answers depend on the scenario. Is the raw JSON payload trusted or arbitrary? Is the enveloping of the raw payload done on behalf of users? This proposal seeks to make all of these post-processing options fully configurable by users, allowing them to tweak the behavior in a way that satisfies their performance, correctness, and security requirements.

In this proposal, raw JSON values will not be reformatted to align with whatever whitespace and indentation settings are specified on JsonWriterOptions, but left as-is. We can provide a future API to reformat raw JSON values. We'll also not encode values by default but can provide API to do it in the future. If users need all three processing options today, the workaround shown above works for that scenario.

Let's go over two of the expected common scenarios, and what API would be configured to enable them.

I have a blob which I think represents JSON content and which I want to envelope, and I need to make sure the envelope & its inner contents remain well-formed

Consider the Azure SDK team which provides a service protocol where the payload content is JSON. Part of the JSON is internal protocol information, and a nested part of the JSON is user provided data. When serializing protocol information, Azure cares about the raw user JSON being structurally valid, and ultimately that the overall JSON payload is valid. They don't wish to alter the formatting of the raw JSON, or preemptively encode it. Given these considerations, they might write their serialization logic as follows:

JsonWriterOptions writerOptions = new() { WriteIndented = true };

using MemoryStream ms = new();
using UtfJsonWriter writer = new(ms, writerOptions);

writer.WriteStartObject();
writer.WriteString("id", protocol.Id);
writer.WriteString("eventType", protocol.EventType);
writer.WriteString("eventTime", DateTimeOffset.UtcNow);
// Write user-provided data.
writer.WritePropertyName("data");
writer.WriteRawValue(protocol.UserBlob);
writer.WriteEndObject();

The resulting JSON payload might look like this:

{
    "id": "a20299ee-f239-42ce-8eee-2da562848fbe",
    "eventType": "Microsoft.Resources.ResourceWriteSuccess",
    "eventTime": "2021-06-10T14:29:10.3363984+00:00",
    "data":{"auth":"{az_resource_mgr_auth}","correlationId":"76d8b695-e98b-4fb6-81c4-1edc1aa22dfc","tenantId":"76d8b695-e98b-4fb6-81c4-1edc1aa22dfc"}
}

Notice that no JsonWriteRawOptions had to be specified.

I have a deliberate sequence of bytes I want to write out on the wire, and I know what I'm doing

Consider a user who needs to format double values differently from the Utf8JsonWriter.WriteNumber[X] methods. Those methods write the shortest possible JSON output required for round-tripping on deserialization. This means that a float or double value that can be expressed simply as 89 would not be written as 89.0. However, our user needs to send numeric JSON content to a service which treats values without decimal points as integers, and values with decimal points as doubles. This distinction is important to our user. The WriteRawValue APIs would allow our custom to format their numeric values as they wish, and write them directly to the destination buffer. In this scenario, their numeric values are trusted, and our user would not want the writer to perform any encoding or structural validation on their raw JSON payloads, to give the fastest possible perf. To satisfy this trusted scenario, our user might write serialization logic as follows:

JsonWriterOptions writerOptions = new() { WriteIndented = true, };

using MemoryStream ms = new();
using UtfJsonWriter writer = new(ms, writerOptions);

writer.WriteStartObject();
writer.WritePropertyName("dataType", "CalculationResults");

writer.WriteStartArray("data");

foreach (CalculationResult result in results)
{
    writer.WriteStartObject();
    writer.WriteString("measurement", result.Measurement);

    writer.WritePropertyName("value");
    // Write raw JSON numeric value
    byte[] formattedValue = FormatNumberValue(resultValue);
    writer.WriteRawValue(formattedValue, skipValidation: true);

    writer.WriteEndObject();
}

writer.WriteEndArray();
writer.WriteEndObject();

The resulting JSON payload might look like this:

{
    "dataType": "CalculationResults",
    "data": [
        {
            "measurement": "Min",
            "value": 50.4
        },
        {
            "measurement": "Max",
            "value": 2
        }
    ]
}

Notes

• The WriteRawValue APIs would work not just for writing raw values as JSON property values (i.e. as POCO properties and dictionary values), but also as root level JSON objects, arrays, and primitives; and as JSON array elements. This functions in the same way as the existing Utf8JsonWriter.WriteXValue methods.

What's next?

• Consider adding support for writing raw property names Consider adding Utf8JsonWriter support for writing raw property names #54005

[ahsonkhan]

Seems like it is "by design"

Yep. In the initial design discussions of Utf8JsonWriter, we had considered adding WriteRaw APIs, but decided against it: https://github.com/dotnet/corefx/issues/33552

there are still scenarios where for performance we want to write raw JSON

We could consider revisiting that decision, if there is evidence of a significant performance benefit in doing so. Do you have some benchmarks for your scenario that would potentially show a win or is there a general concern for too much unnecessary work being done?

In this case, it would be nice to have access to the output stream directly in the custom JsonConverter

One problem with trying to expose the destination is that Utf8JsonWriter could have been constructed to write to different output destinations: Stream (like FileStream) or IBufferWriter<byte> (like ArrayBufferWriter<byte>).

Is your use case specific to uses within JsonConverter? If you are passing the output stream to the JsonSerializer, do you still have access to it?

One of the examples would be using Azure Table Storage, where we want to extract few top-level properties to be PartitionKey/RowKey and store other nested properties in their original JSON form.

Generally speaking, how large are these nested JSON payloads that are stored in their original form, on average (looking for order of magnitude such as 1 KB, 1 MB, 1 GB)?

It's quite often that application layer doesn't care about particular branches of the JSON document and just wants to pass them through to/from the data store (especially when using NoSQL).

What about validation though? Do they care about making sure the JSON is valid though before just writing the string directly calling WriteRawValue or is it assumed the JSON to already be validated elsewhere (like in your case where the JSON retrieved was previously written to storage by the JsonSerializer)?

The best I could find is to use JsonDocument.ParseValue + document.RootElement.GetRawText() when reading it and parse it again into a JsonDocument when writing to just call WriteTo.

Am I missing some APIs?

No, you aren't missing anything. What you found here is the only way you could do this within the JsonConverter. However, one thing you could do to avoid calling GetRawText and an extra re-parse is instead of storing the JSON as string to write later, store the JsonElement from the JsonDocument and call WriteTo on it. Is that feasible for your scenario? It still won't beat the performance of writing the raw bytes directly, but it should improve significantly.

Not knowing the specifics of your scenario, I explored the scenario a bit with some arbitrary JSON data and these are the results:

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.19041
Intel Core i7-6700 CPU 3.40GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.100-alpha1-015914
  [Host]     : .NET Core 5.0.0 (CoreCLR 5.0.19.56303, CoreFX 5.0.19.56306), X64 RyuJIT
  Job-DHBJYG : .NET Core 5.0.0 (CoreCLR 5.0.19.56303, CoreFX 5.0.19.56306), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  

Method	Mean	Error	StdDev	Median	Min	Max	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
RoundTripWithString	95.44 us	1.467 us	1.372 us	95.01 us	93.23 us	98.31 us	1.00	0.00	6.2256	0.8545	-	25.59 KB
RoundTripWithJsonElement	67.49 us	0.543 us	0.454 us	67.47 us	66.65 us	68.14 us	0.71	0.01	4.2725	0.3662	-	17.95 KB
RoundTripRaw	44.97 us	0.688 us	0.610 us	44.83 us	44.31 us	46.29 us	0.47	0.01	7.3853	0.0610	-	30.37 KB
RoundTripWithBytes	92.23 us	1.715 us	1.432 us	91.68 us	90.31 us	94.93 us	0.97	0.02	3.1738	0.2441	-	13.02 KB
RoundTripWithBytesRaw	44.31 us	0.871 us	0.932 us	44.08 us	43.32 us	46.20 us	0.47	0.01	3.1128	0.2441	-	12.95 KB

[nahk-ivanov]

Thanks for taking a stub at this and doing some benchmarking!

At this point it is a general concern for too much unnecessary work being done - this is part of an ASP.NET Core web app hosted in Azure App Service and I'd like to host on as little nodes as possible using smallest SKU possible, so that we scale horizontally, rather than vertically when needed. This looked like a low-hanging fruit to optimize, if all necessary APIs would exist, of course.

Payloads are not big in size (limited by Azure Table Storage entity size anyway), ranging from 1 to 32 KB. But it is an OLTP workload (think online gaming), so we get thousands requests per second and this is not a heavy load yet, of course.

Since serialization/deserialization happens within ASP.NET infrastructure, the easiest would be to hook through JsonConverter. I'll need to check ASP.NET Core APIs to see if I can get a hold of the serializer or request body stream directly to override standard deserialization behavior. I think it might be possible, if I just claim that my action expects a raw binary input, rather than a JSON payload and use JSON reader directly myself. It would be more convenient to have it extract some of the properties automatically though, and only ask me what to with the subset of properties - that's why I was leaning towards JsonConverter approach.

Validation-wise - it is there, but we only need to validate properties at the root level, which we do. Think the following json:

{
  "username": "foo",
  "accessToken": "bar",
  "payload": { "complex": "object" }
}

As long as username and accessToken are valid we don't care if user wants to "shoot himself in a foot" with bogus payload. There are, of course, scenarios where even the payload needs to be validated (e.g. shared with other users), but in this particular case it doesn't matter, as long as overall request size is valid. So what I used to do is map top-level properties and use custom converter on the payload, so that it is preserved in raw form within the application code.

I thought about using JsonElement for the payload property, yeah. This was possible even with JSON.NET as they had similar type, but I don't like serialization-specific types to "leak" into the data model. I'm not happy with using custom attributes, but I can live with it (even though I may not be sleeping well). If I have to use serialization framework-specific types for the properties in the model classes - then it will be even harder to switch from one framework to another.
With attributes I can add custom wrapper types and define converters for them externally to the model, but with public JsonElement Payload { get; set; } it's hard to do anything about it.

Based on your benchmarks I'll probably switch to JsonElement (even though it affects my sleep), but it would be nice if we could easily get performance of RoundTripWithBytesRaw using simple public APIs.

[layomia]

From @declspec in #32849:

I would like to propose an addition to the System.Text.Json.Utf8JsonWriter API which would allow writing existing UTF8-encoded JSON fragments into the current JSON document.

Something like the following:

public  sealed class Utf8JsonWriter {
    public void WriteUtf8JsonValue(ReadOnlySpan<byte> utf8json);

    public void WriteUtf8Json(string propertyName, ReadOnlySpan<byte> utf8json);
    // ... and the rest of the standard overloads for the different types of `propertyName`
}

An example of usage would be:

var utf8json = Encoding.UTF8.GetBytes("{ \"foo\": 42, \"bar\": \"baz\" }").AsSpan();

using (var ms = new MemoryStream()) {
    using (var writer = new Utf8JsonWriter(ms)) {
        writer.WriteStartObject();
        writer.WriteUtf8Json("fragment", utf8json);
        writer.WriteEndObject();
    }

    Debug.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));

    // Expected output:
    // { "fragment": { "foo": 42, "bar": "baz" } }
}

I believe this would be useful when you are trying to augment or wrap existing JSON payloads (i.e. from external services) in a parent document without needing to deserialize and reserialize them.

I have read through all the documentation I can find and as far as I can see there doesn't currently seem to be any way of doing what I'm proposing. If there is already a better way of doing this please let me know.

[mikemcdougall]

What is the workaround?

[george-chakhidze]

What is the workaround?

@mikemcdougall Going back to good ol' Newtonsoft.Json and its JsonWriter.WriteRawValue.

[layomia]

Work-around as described above:

No, you aren't missing anything. What you found here is the only way you could do this within the JsonConverter. However, one thing you could do to avoid calling GetRawText and an extra re-parse is instead of storing the JSON as string to write later, store the JsonElement from the JsonDocument and call WriteTo on it.

string rawJson = ...; // Raw value can also be `byte`s

using (JsonDocument document = JsonDocument.Parse(rawJson))
{
    document.RootElement.WriteTo(writer);
}

[NinoFloris]

We're in a similar situation, we heavily use PG json(b) columns for user defined metadata, our backend reads intentially know nothing about their contents, we'd ideally not pay the price of parsing and serializing something we only pass through.

Api proposal seems solid, good to see it's not based on String.

For large payloads -- LOH allocs are easier to avoid by deserializing, which we're trying to prevent here -- a ReadOnlySequence overload could be of use too.

We're looking at introducing ReadOnlySequence in Npgsql (PG driver) too, meaning we could remove all LOH allocs in an end to end scenario involving potentially large column values.