Description
Bug
The openai-agents SDK's feature to handle structured output using the output_type parameter on agents.Agent fails when the provided Pydantic model utilizes certain common structures, specifically discriminated unions (which translate to oneOf in JSON Schema) or fields with default values.
When such a Pydantic model is provided to output_type, the SDK automatically generates a JSON Schema to pass to the OpenAI Chat Completions API's response_format parameter. However, this generated schema violates the API's validation rules, causing the API to return a 400 Bad Request error and preventing the agent run from succeeding.
We encountered two specific API validation errors related to the SDK's generated schema:
'oneOf' is not permitted within the items definition of an array property. This occurs when output_type is a Pydantic model containing a list of a Union type field using a Discriminator.
'default' is not permitted within a property definition. This occurs when output_type is a Pydantic model where a field has a default value (e.g., my_field: str = "default").
This forces users to abandon the documented output_type mechanism for complex schemas and rely instead on prompt-based JSON generation with manual validation in application code.
Debug information
- Agents SDK version:
v0.0.4 - Python version:
3.11
Repro steps
#!/usr/bin/env python
import asyncio
import os
import sys
from typing import List, Literal, Union, Annotated
from dotenv import load_dotenv
from pydantic import BaseModel, Field, Discriminator
from agents import Agent, Runner
from openai import BadRequestError
load_dotenv()
# --- Pydantic Models Demonstrating Issues ---
# 1. Models for 'oneOf is not permitted' issue
class SpecificDataA(BaseModel):
item_type: Literal["type_a"]
value_a: str
class SpecificDataB(BaseModel):
item_type: Literal["type_b"]
value_b: int
AnySpecificData = Annotated[
Union[SpecificDataA, SpecificDataB], Discriminator("item_type")
]
class OutputWithUnionList(BaseModel):
results: List[AnySpecificData]
# 2. Model for 'default is not permitted' issue
class OutputWithDefault(BaseModel):
required_field: str
field_with_default: str = Field("some_default") # Default value
# --- Agent Definitions ---
MODEL_TO_USE = "gpt-4o-mini" # Or any model supporting JSON mode
AGENT_WITH_UNION = Agent(
name="UnionAgent",
instructions=f"Output data matching {OutputWithUnionList.__name__}.",
model=MODEL_TO_USE,
output_type=OutputWithUnionList, # Triggers 'oneOf' schema error
)
AGENT_WITH_DEFAULT = Agent(
name="DefaultAgent",
instructions=f"Output data matching {OutputWithDefault.__name__}.",
model=MODEL_TO_USE,
output_type=OutputWithDefault, # Triggers 'default' schema error
)
# --- Test Runner Function ---
async def run_agent_test(agent: Agent, test_input: str):
"""Runs agent, expecting BadRequestError due to schema issues."""
print(f"\n--- Running Test: {agent.name} ---") # Simplified print
try:
await Runner.run(agent, input=test_input)
print(
f" [UNEXPECTED SUCCESS] Agent {agent.name} completed."
) # Should not happen
except BadRequestError as e:
# Catch and report the specific error we expect
print(f" [EXPECTED FAILURE] Code: {e.status_code}, Message: {e.message}")
# Let other unexpected errors propagate naturally
# --- Main Execution ---
async def main():
print("Starting OpenAI Agents SDK Bug Reproduction Script...")
# Removed explanation print statement
await run_agent_test(AGENT_WITH_UNION, "Generate one type_a and one type_b item.")
await run_agent_test(AGENT_WITH_DEFAULT, "Generate data with required field.")
print("\nScript finished.")
if __name__ == "__main__":
asyncio.run(main())
Expected behavior
The Runner.run call should succeed without API errors. The result.final_output should be an instance of the Pydantic model specified in output_type.
Actual Result:
The Runner.run call fails, raising an openai.BadRequestError from the API call made internally by the SDK. The specific error message depends on which model structure caused the issue:
For AGENT_WITH_UNION (using OutputWithUnionList):
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid schema for response_format 'final_output': In context=('properties', 'results', 'items'), 'oneOf' is not permitted.", ...}}
For AGENT_WITH_DEFAULT (using OutputWithDefault):
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid schema for response_format 'final_output': In context=('properties', 'field_with_default'), 'default' is not permitted.", ...}}