[AI Evaluation] EquivalenceEvaluator uses invalid MaxOutputTokens for Azure OpenAI model

[NAvanDijk]

Description

When attempting to use the EquivalenceEvaluator, my tests throw an exception from Azure:

Azure.RequestFailedException : Invalid 'max_completion_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead.
Status: 400 (Bad Request)
ErrorCode: integer_below_min_value

Content: 
{ 
    "error": { 
        "message": "Invalid 'max_completion_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead.",
        "type": "invalid_request_error", 
        "param": "max_completion_tokens", 
        "code": "integer_below_min_value" 
    }
}

The limit has been increased to 5 in a PR to fix another issue.

Reproduction Steps

var messages = new List<ChatMessage>
{
    new(ChatRole.System, "Your are a helpful assistant."),
    new(ChatRole.Assistant, "What's the 3rd planet from the sun?")
};

var response = new ChatResponse(new ChatMessage(ChatRole.Assistant, "The Earth is the 3rd planet."));

var chatConfig = new ChatConfiguration(chatClient);

var equivalenceEvaluatorContext = new EquivalenceEvaluatorContext("The 3rd planet from the sun is the Earth.");

var equivalenceEvaluator = new EquivalenceEvaluator();
var evaluationResult = await equivalenceEvaluator.EvaluateAsync(messages, response, chatConfig, additionalContext: [equivalenceEvaluatorContext]);

Debug.WriteLine(evaluationResult.Metrics.Single().Value.Diagnostics?.Single().Message);

Expected behavior

The evaluator should complete without any exceptions.

Actual behavior

Azure returns a bad request with the following message: Invalid 'max_completion_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead.

Regression?

No response

Known Workarounds

No response

Configuration

Using dotnet 10.0.100-rc.2.25502.107. I am getting the IChatClient from ChatClientAgent.ChatClient with the Microsoft.Agents.AI nuget package. I am using a deployed gpt-4o-mini model in Azure AI Foundry.

Other information

No response