Error with MEAI 9.4.0-preview.1.25175.4

[JamesNK]

Description

I tried out 9.4.0-preview.1.25175.4 and I got this error:

fail: Microsoft.Extensions.AI.LoggingChatClient[1784604714]
      GetStreamingResponseAsync failed.
      System.ClientModel.ClientResultException: HTTP 400 (invalid_request_error: invalid_function_parameters)
      Parameter: tools[3].function.parameters

      Invalid schema for function 'GetLogs': In context=('properties', 'name'), 'default' is not permitted.
         at OpenAI.ClientPipelineExtensions.ProcessMessageAsync(ClientPipeline pipeline, PipelineMessage message, RequestOptions options)
         at OpenAI.Chat.ChatClient.CompleteChatAsync(BinaryContent content, RequestOptions options)
         at OpenAI.Chat.ChatClient.<>c__DisplayClass10_0.<<CompleteChatStreamingAsync>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at OpenAI.AsyncSseUpdateCollection`1.GetRawPagesAsync()+MoveNext()
         at OpenAI.AsyncSseUpdateCollection`1.GetRawPagesAsync()+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at System.ClientModel.AsyncCollectionResult`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
         at System.ClientModel.AsyncCollectionResult`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
         at System.ClientModel.AsyncCollectionResult`1.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at Microsoft.Extensions.AI.OpenAIModelMappers.FromOpenAIStreamingChatCompletionAsync(IAsyncEnumerable`1 updates, CancellationToken cancellationToken)+MoveNext()
         at Microsoft.Extensions.AI.OpenAIModelMappers.FromOpenAIStreamingChatCompletionAsync(IAsyncEnumerable`1 updates, CancellationToken cancellationToken)+MoveNext()
         at Microsoft.Extensions.AI.OpenAIModelMappers.FromOpenAIStreamingChatCompletionAsync(IAsyncEnumerable`1 updates, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at Microsoft.Extensions.AI.LoggingChatClient.GetStreamingResponseAsync(IEnumerable`1 messages, ChatOptions options, CancellationToken cancellationToken)+MoveNext()

GetLogs is a function with a name parameter and it allows null.

[Description("Get logs.")]
private async Task<string?> GetLogs(
    [Description("The name. Optional.")]
    string? name = null,
    CancellationToken cancellationToken = default)
{
}

I haven't made any changes to my code other than updating to 9.4.0-preview.1.25175.4.

Endpoint is https://api.openai.com/v1

Reproduction Steps

Call GetStreamingResponseAsync

Expected behavior

AI response returned

Actual behavior

See error above.

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

[stephentoub]

@eiriktsarpalis, can you take a look? I'm wondering if this is a bug in our schema generation, or if we need to flip openai's strict noise back off.

[eiriktsarpalis]

The error message seems to suggest that default keywords are not permitted, but I see nothing in the OpenAI documentation specifically banning default. We have been using the default keyword to populate default values in optional parameters -- we could always try to remove that using the transformer methods if need be.

[stephentoub]

Maybe for AIJsonUtilities schema purposes both optional and nullable parameters in C# should just manifest as non-required? I realize it's not the letter of the schema law, but it seems much more common to how things are represented, and we're already in opinion territory in how C# functions are translated to schema.

[stephentoub]

cc: @SteveSandersonMS

[JamesNK]

Before: 9.3.0-preview.1.25161.3
After: 9.4.0-preview.1.25175.4

I assume something changed between those versions in the request sent to OpenAI that caused it to reject the request.

[stephentoub]

Before: 9.3.0-preview.1.25161.3 After: 9.4.0-preview.1.25175.4

I assume something changed between those versions in the request sent to OpenAI that caused it to reject the request.

Yes, this is what I was referring to around "strict". A change was made to opt-in by default to OpenAI's "strict" schema setting, which guarantees it'll abide by the schema but also has stricter constraints on what the schema is.

When you construct your AIFunction, try passing in AdditionalProperties["Strict"] = false... does that "fix" it? e.g.

AIFunctionFactory.Create(yourMethod, new() { AdditionalProperties = new Dictionary<string, object?>() { ["Strict"] = false } })

[JamesNK]

Yes that fixes it.

[eiriktsarpalis]

I see two options:

1. We update the AIJsonUtilities schema exporter to strip out default keywords. This does not guarantee that other keywords will not be rejected further down the line though.
2. Do the above but also revert strict mode to being an opt-in setting.