Repository navigation
[Microsoft.Extensions.AI] AIFunctionFactory does not tolerate a JsonElement of ValueKind String as a JSON-stringified argument (gap in #6572) #7626
Description
Package: Microsoft.Extensions.AI.Abstractions 10.4.1 (code unchanged on main at tag v10.4.1)
Summary
#6572 ("AIFunctionFactory: tolerate JSON string function parameters", fixing #6544) made
AIFunctionFactory's parameter marshaller tolerant of a parameter whose value is a JSON string
wrapping the real argument (e.g. "{\"a\":1}" for an object parameter). That tolerance
(IsPotentiallyJson + MarshallViaJsonRoundtrip) is only reached when the argument value is a
raw .NET string.
When the same JSON-stringified object arrives as a JsonElement whose ValueKind == String,
the marshaller takes the earlier JsonElement arm and calls
JsonSerializer.Deserialize(element, typeInfo) directly, which throws before the tolerant path runs.
This is the exact shape produced when a model double-encodes a tool-call argument (emits
"input":"{...}" instead of "input":{...}) and the arguments JSON is parsed into JsonElements
(as FunctionInvokingChatClient-backed pipelines do). The failure is intermittent from the model's
side — the model usually emits the correct nested object and only occasionally stringifies it — but
whenever that shape occurs, the deserialization failure below is deterministic. A raw string with the
identical content is handled fine; a JsonElement string is not.
JsonElement element => JsonSerializer.Deserialize(element, typeInfo), // throws for ValueKind == String
JsonDocument doc => JsonSerializer.Deserialize(doc, typeInfo),
JsonNode node => JsonSerializer.Deserialize(node, typeInfo),
_ => MarshallViaJsonRoundtrip(value), // <- IsPotentiallyJson tolerance lives hereRepro (net9.0, Microsoft.Extensions.AI.Abstractions 10.4.1)
using System.Text.Json;
using Microsoft.Extensions.AI;
public sealed class Input { public string A { get; init; } = ""; public string B { get; init; } = ""; }
var func = AIFunctionFactory.Create((Input input) => "ok", new AIFunctionFactoryOptions { Name = "f" });
string inner = "{\"a\":\"x\",\"b\":\"y\"}";
// A: JsonElement of ValueKind String -> THROWS
await Invoke(JsonSerializer.SerializeToElement(inner));
// B: raw .NET string (same content) -> OK (tolerated by #6572)
await Invoke(inner);
// C: JsonElement of ValueKind Object -> OK
await Invoke(JsonSerializer.Deserialize<JsonElement>(inner));
async Task Invoke(object value)
{
try { Console.WriteLine("OK: " + await func.InvokeAsync(new AIFunctionArguments { ["input"] = value })); }
catch (Exception ex) { Console.WriteLine(ex.GetType().Name + ": " + ex.Message); }
}Output:
JsonException: The JSON value could not be converted to Input. Path: $ | LineNumber: 0 | BytePositionInLine: 17.
OK: ok
OK: ok
Expected
A JsonElement of ValueKind == String whose content decodes to a valid argument should be
tolerated the same way a raw string is (parity with #6572), rather than throwing JsonException.
Actual
System.Text.Json.JsonException: The JSON value could not be converted to <T>. Path: $ thrown from
ObjectDefaultConverter<T>.OnTryRead via the GetParameterMarshaller delegate. In a
FunctionInvokingChatClient pipeline this surfaces as a failed tool invocation.
Why this affects real first-party clients (not a synthetic-only case)
The failing shape — an argument stored as a JsonElement of ValueKind == String — is exactly what
first-party IChatClient bridges produce, because they parse tool-call arguments into an
IDictionary<string, object?> / Dictionary<string, object?>, and System.Text.Json boxes every
value in such a dictionary as a JsonElement. So when a model double-encodes a tool argument, that
value arrives as a JsonElement{ValueKind: String} and hits the throwing arm above — the #6572
tolerance (raw string only) never runs.
Two public first-party bridges do exactly this today:
-
Microsoft.Extensions.AI.OpenAI —
OpenAIResponsesChatClientmaps aFunctionCallResponseItem
viaOpenAIClientExtensions.ParseCallContent(...), which calls
FunctionCallContent.CreateFromParsedArguments(json, ..., json => JsonSerializer.Deserialize(json, OpenAIJsonContext.Default.IDictionaryStringObject)).
Target typeIDictionary<string, object>⇒ values areJsonElement.
https://github.com/dotnet/extensions/blob/main/src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs
(thecase FunctionCallResponseItem:arm) -
Official Anthropic .NET SDK —
AnthropicClientExtensionsmaps aToolUseBlockvia
FunctionCallContent.CreateFromParsedArguments(element.GetRawText(), ..., json => JsonSerializer.Deserialize(json, ...GetTypeInfo(typeof(Dictionary<string, object?>)))).
Target typeDictionary<string, object?>⇒ values areJsonElement.
https://github.com/anthropics/anthropic-sdk-csharp/blob/main/src/Anthropic/AnthropicClientExtensions.cs
(thecase ToolUseBlock toolUse:arm inContentBlockValueToAIContent)
In both, a double-encoded argument reaches AIFunctionFactory's marshaller as a
JsonElement{ValueKind: String} and throws. Because both first-party bridges share this shape, a fix
in GetParameterMarshaller repairs every provider at once. Note the divergence with Semantic Kernel
(the original #6572 reporter): SK's KernelArguments carry raw .NET string values — the one shape
#6572 already tolerates — so the same logical defect was fixed for SK but not for the
JsonElement-string shape the OpenAI/Anthropic bridges produce.
Suggested fix
In the JsonElement arm of GetParameterMarshaller, when element.ValueKind == JsonValueKind.String
and the target type is not string, attempt to re-parse element.GetString() as JSON before
deserializing — mirroring the existing IsPotentiallyJson / MarshallViaJsonRoundtrip handling used
for raw strings.
Activity
jeffhandley commentedon Jul 15, 2026
As @eiriktsarpalis pointed out in #7630 (comment), even if we did tolerate the outer parameter, that would leave open the possibility of a nested part of the object being encoded, and we certainly don't want to get into recursive forgiveness.
Rather than trying to tolerate malformed arguments, we should look for opportunities to improve the error handling here so that the LLM can self-correct.
@flaviocdc After the exception is thrown, it's unclear to me what happens to the end-to-end scenario thereafter. Does that error make it back to the model? Is the model given a chance to retry the tool call? Is it given enough information to understand why the tool call failed? Is it able to self-correct?
If the tool call fails and the model isn't given the opportunity or information needed to retry with corrected formatting, I'd be supportive of addressing that gap rather than forgiveness on the parameter encoding.
flaviocdc commentedon Jul 16, 2026
Thanks @eiriktsarpalis @jeffhandley — that reasoning makes sense to me, and I agree the marshaller isn't the right place to be forgiving of malformed arguments. The nested double-encoding gap and the risk to types that legitimately serialize as JSON strings (
Guid,DateTimeOffset, custom converters) are convincing, and "we can't anticipate every malformation an LLM might produce" is fair.One thing I looked at on the error-handling angle: today the exception is caught, turned into a
FunctionResultContent, and the loop continues back to the model — but with the defaultIncludeDetailedErrors = falsethe model just receives"Error: Function failed.", and in the trace I looked at it didn't try to self-correct, it just moved on. So the retry path exists, but the signal the model gets is very thin.I don't want to prescribe a fix here — you both have much better context on the intended design. Are there better ways you'd want to see this handled?
flaviocdc commentedon Jul 16, 2026
Here's how the LLM chat thread looks like:
mauri0686 commentedon Sep 24, 2026
Following up on the discussion above: I opened #7779, which addresses the first half of the feedback gap without tolerating malformed arguments. The marshaller's
JsonExceptionnow names the parameter and, when a JSON string arrives where an object or array is expected, says so explicitly. That helps logs and telemetry, and it reaches the model whenIncludeDetailedErrorsis enabled.The second half remains: with the default
IncludeDetailedErrors = false, the model still only receivesError: Function failed.. To close that,FunctionInvokingChatClientwould need to tell argument binding failures (caused by the model's input, with library-authored messages) apart from exceptions thrown by the function itself. As far as I can tell that requires a new public type, so I didn't want to go further without your input. @jeffhandley @eiriktsarpalis, would you be open to an API proposal along those lines, or do you prefer a different approach?