BadRequestError: 400 - max_tokens or model output limit reached when using beta.chat.completions.parse() with Azure OpenAI GPT-5, even without setting max_tokens

[Kevv-J]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

When calling client.beta.chat.completions.parse() for structured output on an Azure-hosted GPT-5 deployment, the API returns a 400 BadRequestError:

openai.BadRequestError: Error code: 400 - {                                                                                                                                  
  'error': {
    'message': 'Could not finish the message because max_tokens or model output limit was reached. Please try again with higher max_tokens.',
    'type': 'invalid_request_error',
    'param': None,
    'code': None
  }
}

This occurs even though neither max_tokens nor max_completion_tokens is set anywhere in the request.

This is related to #2046, which was closed with the suggestion to use max_completion_tokens instead of max_tokens. That resolution does not apply here, this issue occurs when no token limit parameter is passed at all. The error message is therefore misleading: it implies the caller set a limit that was too low, when in fact no limit was set.

Expected Behavior

No token limit is enforced when neither max_tokens nor max_completion_tokens is provided, consistent with how chat.completions.create() behaves.

Actual Behavior

A 400 BadRequestError is raised claiming the model output limit was reached, despite no limit being set by the caller.

To Reproduce

1. Create an AsyncAzureOpenAI client pointing to a GPT-5 Azure deployment
2. Call client.beta.chat.completions.parse() with a Pydantic model as response_format
3. Pass reasoning_effort but do not pass max_tokens or max_completion_tokens
4. Use a moderately complex Pydantic schema (e.g. nested models)

Code snippets

from openai import AsyncAzureOpenAI

client = AsyncAzureOpenAI(
    azure_endpoint="<AZURE_GPT5_ENDPOINT>",
    azure_deployment="<AZURE_GPT5_DEPLOYMENT>",
    api_version="<API_VERSION>",
    api_key="<API_KEY>",
)

completion = await client.beta.chat.completions.parse(
    model="<model_name>",
    messages=[
        {"role": "developer", "content": system_prompt},
        {"role": "user", "content": user_prompt},
    ],
    response_format=MyPydanticOutputClass,  # Pydantic model for structured output
    reasoning_effort="minimal",
    # max_tokens and max_completion_tokens are intentionally NOT set
)

result = completion.choices[0].message.parsed

OS

Ubuntu 24.04.2 LTS

Python version

3.11.13

Library version

openai 1.75.0

[xXMrNidaXx]

This looks like an Azure OpenAI-specific default behavior rather than an SDK bug. A few things happening here:

Root Cause Analysis:

Azure OpenAI deployments often have model-level defaults for max_tokens that differ from the OpenAI API. When using beta.chat.completions.parse(), the SDK may not be explicitly setting max_completion_tokens, which causes Azure to fall back to a model default that's lower than needed for structured output generation.

Why structured output is particularly affected:

With response_format={"type": "json_schema", ...}, the model needs to generate valid JSON matching your schema. If the schema is complex or includes many fields, the default token limit may be insufficient to complete the JSON structure, causing the 400 error.

Workarounds:

1. Explicitly set max_completion_tokens:

   response = client.beta.chat.completions.parse(
       model="gpt-5",
       messages=messages,
       response_format=YourSchema,
       max_completion_tokens=4096  # or higher based on schema complexity
   )

2. Check Azure deployment settings:
   In Azure Portal → your OpenAI resource → Model deployments, verify if there's a default max_tokens configured at the deployment level.

3. Use the non-beta endpoint with manual parsing:
   The regular chat.completions.create() endpoint might handle Azure defaults differently.

For the SDK maintainers:

Consider adding Azure-specific handling that explicitly passes max_completion_tokens=None (or a reasonable default like 4096) when it's not specified, to prevent Azure from applying its own default. The behavior difference between Azure and OpenAI API endpoints is a common pain point.

---

Corey @ RevolutionAI — we help teams debug and deploy LLM integrations at scale

[xXMrNidaXx]

Hit this exact issue with Azure GPT-5 + structured outputs. Here's what worked as a workaround:

Workaround: Explicitly set max_completion_tokens

response = await client.beta.chat.completions.parse(
    model='gpt-5',
    messages=messages,
    response_format=YourPydanticModel,
    max_completion_tokens=16384,  # Set explicitly high
    reasoning_effort='medium'
)

Why this happens:
Azure's default token limits are more restrictive than OpenAI's direct API. When using structured outputs + reasoning, the model needs extra tokens for the reasoning trace + JSON schema compliance. Without an explicit limit, Azure seems to apply a low default.

Additional tips:

• Start with max_completion_tokens=16384 (or higher for complex schemas)
• Monitor actual token usage to tune down if needed
• Simpler Pydantic schemas = fewer tokens needed

Would be great if the SDK could auto-detect Azure and set sensible defaults, or at least surface a clearer error message!

[xXMrNidaXx]

Analysis and Workarounds

This looks like Azure's GPT-5 deployment has a default max_completion_tokens limit that's lower than what the reasoning model + structured output combination needs.

Why This Happens

When reasoning_effort is set (even to "minimal"), GPT-5 models allocate tokens for internal reasoning before generating the structured response. Combined with a complex Pydantic schema, the total token requirement often exceeds Azure's default limits.

The error message is misleading because:

1. Azure's backend sets an implicit limit even when you don't specify one
2. The limit is lower than OpenAI's direct API defaults

Workarounds

Option 1: Explicitly set a high max_completion_tokens

completion = await client.beta.chat.completions.parse(
    model="<model_name>",
    messages=[...],
    response_format=MyPydanticOutputClass,
    reasoning_effort="minimal",
    max_completion_tokens=16384,  # Explicit high limit
)

Option 2: Simplify the Pydantic schema

Complex nested models with many fields require more tokens. Consider:

• Flattening nested structures
• Using Optional fields strategically
• Splitting into multiple smaller API calls

Option 3: Use non-beta chat.completions.create

completion = await client.chat.completions.create(
    model="<model_name>",
    messages=[...],
    response_format={"type": "json_object"},
    max_completion_tokens=16384,
)
# Manual Pydantic validation
result = MyPydanticOutputClass.model_validate_json(
    completion.choices[0].message.content
)

Option 4: Reduce reasoning overhead

# Remove reasoning_effort entirely if not needed
completion = await client.beta.chat.completions.parse(
    model="<model_name>",
    messages=[...],
    response_format=MyPydanticOutputClass,
    # reasoning_effort removed
    max_completion_tokens=8192,
)

Suggested SDK Improvement

The SDK could catch this error pattern and surface a more helpful message:

except BadRequestError as e:
    if "max_tokens or model output limit" in str(e):
        raise BadRequestError(
            f"Azure deployment hit implicit token limit. "
            f"Try adding max_completion_tokens=16384. Original: {e}"
        )

---

Hit this while building structured extraction pipelines for enterprise clients. Azure's different defaults from OpenAI direct API are a common source of confusion.