Feature: Add OpenAIAgent backed by OpenAI Response API #6418
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6418 +/- ##
==========================================
+ Coverage 79.53% 79.67% +0.14%
==========================================
Files 225 226 +1
Lines 16650 16933 +283
==========================================
+ Hits 13242 13491 +249
- Misses 3408 3442 +34
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
python/packages/autogen-ext/src/autogen_ext/agents/openai/_openai_agent.py
Outdated
Show resolved
Hide resolved
| if self._tools: | ||
| api_params["tools"] = self._tools | ||
| if self._json_mode: | ||
| api_params["text.format"] = "json" # Responses API uses text.format for structured outputs |
There was a problem hiding this comment.
json_schema is recommended by official doc.
python/packages/autogen-ext/src/autogen_ext/agents/openai/_openai_agent.py
Show resolved
Hide resolved
| if self._seed is not None: | ||
| api_params["seed"] = self._seed | ||
| if self._tools: | ||
| api_params["tools"] = self._tools | ||
| if self._json_mode: | ||
| api_params["text.format"] = "json" # Responses API uses text.format for structured outputs | ||
| api_params["json_schema"] = {"type": "object"} |
There was a problem hiding this comment.
If you want to turn of JSON mode, this line should be changed to
api_params["text"] = { "format": { "type": "json_object" } }Please refer https://platform.openai.com/docs/guides/structured-outputs#structured-outputs-vs-json-mode.
Why not considering structured outputs in here?
api_params["text"] = { "format": { "type": "json_schema", "strict": true, "schema": "..." } }
| history: List[Dict[str, Any]] = Field(default_factory=list) | ||
|
|
||
|
|
||
| class OpenAIAgentConfig(BaseModel): |
There was a problem hiding this comment.
- Default value of
temperatureparameter is1, and the range should be between 0 and 2. - Default value of
storeparameter isTrue. - Default value of
truncationparameter is"disabled". - The OpenAI Response API doesn't have the
seedparameter.
| if isinstance(part, TextMessage): | ||
| content_parts.append({"type": "text", "text": str(part.content)}) | ||
| elif isinstance(part, ImageMessage): | ||
| image_url = str(part.content) |
There was a problem hiding this comment.
According to https://github.com/microsoft/autogen/pull/6418/files#diff-81b975348a92efc2e2df60eade414fd6cddee0ff1e0bd8acd5481eeed70da0faR712-R715, part.content can be a base64 string.
| if self._message_history: | ||
| input_messages.extend(self._message_history) | ||
|
|
||
| for message in messages: |
There was a problem hiding this comment.
I think you can optimize 560-578 lines for better code readability.
| return "[image]" | ||
|
|
||
| def to_text(self) -> str: | ||
| return f"[Image: {self.content[:30]}...]" if len(self.content) > 30 else f"[Image: {self.content}]" |
There was a problem hiding this comment.
What are the hardcoded 30 lengths? I think it would be useful to test the image functionality and some of these edge cases.
| if self._tools: | ||
| api_params["tools"] = self._tools | ||
| if self._json_mode: | ||
| api_params["json_schema"] = {"type": "object"} |
There was a problem hiding this comment.
It seems like this should be:
api_params["response_format"] = { "type": "json_object" }
Base on:
https://platform.openai.com/docs/api-reference/responses/create
|
ack all comments. |
@jay-thakur are you working on addressing these? If not I'm happy to pick them up |
|
yes, I am. will address all these comments and raise updated PR. @bassmang |
Why are these changes needed?
This PR introduces a new
OpenAIAgentimplementation that uses the OpenAI Response API as its backend. The OpenAI Assistant API will be deprecated in 2026, and the Response API is its successor. This change ensures our codebase is future-proof and aligned with OpenAI’s latest platform direction.Motivation
AssistantAgentin AgentChat, but is implemented directly on top of the OpenAI Response API.Key Changes
OpenAIAgent, a stateful agent that interacts with the OpenAI Response API.AssistantAgentin AgentChat, ensuring a smooth migration path.openaiPython library for all API interactions.Migration Path
OpenAIAgentto ensure long-term compatibility.References
Related issue number
Closes #6032
Checks