Add OpenAI Responses API #1256

Kludex · 2025-03-27T10:52:49Z

Closes Support OpenAI new response API format and new tools #1099

I would like on a follow-up PR support the native OpenAI Responses function tools, and all parameters via OpenAIModelSettings.

github-actions · 2025-03-27T11:13:31Z

Docs Preview

commit:	`deb5aed`
Preview URL:	https://be50e4e1-pydantic-ai-previews.pydantic.workers.dev

samuelcolvin

Also needs to be added to docs.

samuelcolvin · 2025-03-28T09:59:53Z

pydantic_ai_slim/pydantic_ai/models/openai.py

@@ -399,6 +404,298 @@ async def _map_user_prompt(part: UserPromptPart) -> chat.ChatCompletionUserMessa
        return chat.ChatCompletionUserMessageParam(role='user', content=content)


+@dataclass(init=False)
+class OpenAIResponsesModel(Model):
+    """A model that uses the OpenAI Responses API."""


needs a better docstring including:

link to some resource about the responses API

why you might use it

brief example of how to use it

It doesn't seem we are adding examples in the analogous classes, but I added a more complete docstring.

samuelcolvin · 2025-03-28T10:00:45Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+        self,
+        model_name: OpenAIModelName,
+        *,
+        provider: Literal['openai', 'deepseek', 'azure'] | Provider[AsyncOpenAI] = 'openai',


only partially related, but should we add ollama or local here? Do all the local models default to the same port?

We should, we just didn't yet. We need to implement their providers.

samuelcolvin · 2025-03-28T10:02:27Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+        # standalone function to make it easier to override
+        if not tools:
+            tool_choice: Literal['none', 'required', 'auto'] | None = None
+        elif not model_request_parameters.allow_text_result:
+            tool_choice = 'required'
+        else:
+            tool_choice = 'auto'


should we move this into a separate function and reuse it?

I avoided, because I have a PR to do it already.

samuelcolvin · 2025-03-28T10:02:39Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+        reasoning_effort = model_settings.get('openai_reasoning_effort', NOT_GIVEN)
+        if not isinstance(reasoning_effort, NotGiven):
+            reasoning = Reasoning(effort=reasoning_effort)
+        else:
+            reasoning = NOT_GIVEN


Hmmm... Actually... I can't make this work type wise 🤔

It seems the overload from OpenAI doesn't work when I do this:

@staticmethod def _get_reasoning(model_settings: OpenAIModelSettings) -> Reasoning | None | NotGiven: reasoning_effort = model_settings.get('openai_reasoning_effort', NOT_GIVEN) if not isinstance(reasoning_effort, NotGiven): return Reasoning(effort=reasoning_effort) return NOT_GIVEN

And assign that to reasoning.

samuelcolvin · 2025-03-28T10:06:21Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+            elif isinstance(chunk, responses.ResponseContentPartAddedEvent):
+                pass  # there's nothing we need to do here
+
+            elif isinstance(chunk, responses.ResponseContentPartDoneEvent):
+                pass  # there's nothing we need to do here
+
+            elif isinstance(chunk, responses.ResponseCreatedEvent):
+                pass  # there's nothing we need to do here


What are these parts? Are we sure we shouldn't use them, or if we don't know how to use them, then error.

Could you add a comment explaining what each part is to make it easy to understand in future.

I specifically took @dmontagu time yesterday on this part of the code. We went through the events together.

please can you add a TODO to review and either warn or error if we get parts we don't yet support.

please can you add a TODO to review and either warn or error if we get parts we don't yet support.

We already have a warning below on events that we don't do anything. Those events here are the ones we checked and we are not supposed to do anything. At least, there's no need given the other events we already handle.

samuelcolvin · 2025-03-28T10:06:28Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+                pass  # there's nothing we need to do here
+
+            elif isinstance(chunk, responses.ResponseFunctionCallArgumentsDeltaEvent):
+                maybe_event = self._parts_manager.handle_tool_call_delta(


comment explaining what's going on.

It will be hard for me to explain this, my understanding here is shallow. That's why I took David's time to help me in the streaming part.

I just know the obvious... If we see chunk of type x, then we apply the delta...

samuelcolvin · 2025-03-28T10:06:36Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+                pass  # there's nothing we need to do here
+
+            elif isinstance(chunk, responses.ResponseInProgressEvent):
+                pass  # there's nothing we need to do here


again comment.

samuelcolvin · 2025-03-28T10:07:00Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+
+            elif isinstance(chunk, responses.ResponseOutputItemDoneEvent):
+                # Note: We only need this if the tool call deltas don't include the final info:
+                # if isinstance(chunk.item, responses.ResponseFunctionToolCall):


should we error here?

No, it looks like the tool call deltas do include the final info. We left that there just in case it ends up being possible in some way to cause OpenAI to not send this info. But for now I think this is the right implementation. I'll add a note above the commented out code explaining this more clearly

can we remove the commented out code then.

pydantic_ai_slim/pydantic_ai/models/openai.py

hyperlint-ai · 2025-03-28T10:36:54Z

PR Change Summary

Added support for OpenAI's Responses API in the documentation, including usage examples and links to relevant resources.

Introduced the OpenAI Responses API documentation section
Provided examples of using the OpenAIResponsesModel class
Included links to OpenAI's built-in tools and API documentation

Modified Files

docs/models.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

samuelcolvin · 2025-03-28T12:09:44Z

pydantic_ai_slim/pydantic_ai/models/openai.py

    """
    Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning).
+


@Kludex we need to add support for setting hosted tools here.

samuelcolvin · 2025-03-31T11:10:38Z

docs/models.md

+
+model = OpenAIResponsesModel('gpt-4o')
+agent = Agent(model)
+...


This example should be complete, if minimal.

Well... There are 4 examples above that are exactly the same as this one. 🤔

Should I update all of them? Did you see them?

samuelcolvin · 2025-03-31T11:10:54Z

docs/models.md

+- [Web search](https://platform.openai.com/docs/guides/tools-web-search)
+- [File search](https://platform.openai.com/docs/guides/tools-file-search)
+- [Computer use](https://platform.openai.com/docs/guides/tools-computer-use)
+


we need a note saying this is not yet supported in PydanticAI, will be soon.

samuelcolvin · 2025-03-31T11:13:12Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+
+            elif isinstance(chunk, responses.ResponseOutputItemDoneEvent):
+                # Note: We only need this if the tool call deltas don't include the final info:
+                # if isinstance(chunk.item, responses.ResponseFunctionToolCall):


can we remove the commented out code then.

samuelcolvin · 2025-03-31T11:13:55Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+            elif isinstance(chunk, responses.ResponseContentPartAddedEvent):
+                pass  # there's nothing we need to do here
+
+            elif isinstance(chunk, responses.ResponseContentPartDoneEvent):
+                pass  # there's nothing we need to do here
+
+            elif isinstance(chunk, responses.ResponseCreatedEvent):
+                pass  # there's nothing we need to do here


please can you add a TODO to review and either warn or error if we get parts we don't yet support.

Kludex force-pushed the add-openai branch from 1910354 to 7416764 Compare March 27, 2025 11:06

Add OpenAI Responses API

00178b1

Kludex force-pushed the add-openai branch from 7416764 to 00178b1 Compare March 27, 2025 11:08

Kludex and others added 11 commits March 27, 2025 12:23

Merge remote-tracking branch 'origin/main' into add-openai

d7a57bd

Add tests

6b4da84

Add more tests

b6c444c

Add 100% coverage

c3cc473

Add streaming response

67a30e7

Fix some bugs

600553c

Minor tweaks

98da00b

fix tests

0ff6da3

Merge branch 'main' into add-openai

3f5b20c

100% coverage

9f52eb0

Add a pragma

69782bc

samuelcolvin reviewed Mar 28, 2025

View reviewed changes

Apply comments

3e1a3fe

samuelcolvin reviewed Mar 28, 2025

View reviewed changes

Kludex added 3 commits March 28, 2025 14:37

Add more code related to stream events

119030d

Merge remote-tracking branch 'origin/main' into add-openai

4534410

Add pragma

d40a2ef

Kludex requested review from dmontagu and samuelcolvin March 31, 2025 09:04

samuelcolvin reviewed Mar 31, 2025

View reviewed changes

Kludex added 3 commits March 31, 2025 14:56

Apply comments

deb5aed

Merge branch 'main' into add-openai

00f9438

Merge branch 'main' into add-openai

f91afd0

Kludex enabled auto-merge (squash) March 31, 2025 14:27

Kludex merged commit bea1605 into main Mar 31, 2025
13 checks passed

Kludex deleted the add-openai branch March 31, 2025 14:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add OpenAI Responses API #1256

Add OpenAI Responses API #1256

Kludex commented Mar 27, 2025 •

edited

Loading

github-actions bot commented Mar 27, 2025 •

edited

Loading

samuelcolvin left a comment

samuelcolvin Mar 28, 2025

Kludex Mar 28, 2025

samuelcolvin Mar 28, 2025

Kludex Mar 28, 2025

samuelcolvin Mar 28, 2025

Kludex Mar 28, 2025

samuelcolvin Mar 28, 2025

Kludex Mar 28, 2025

samuelcolvin Mar 28, 2025

Kludex Mar 28, 2025

samuelcolvin Mar 31, 2025

Kludex Mar 31, 2025

samuelcolvin Mar 28, 2025

Kludex Mar 28, 2025

samuelcolvin Mar 28, 2025

samuelcolvin Mar 28, 2025

dmontagu Mar 28, 2025 •

edited

Loading

samuelcolvin Mar 31, 2025

hyperlint-ai bot commented Mar 28, 2025

samuelcolvin Mar 28, 2025

samuelcolvin Mar 31, 2025

Kludex Mar 31, 2025 •

edited

Loading

samuelcolvin Mar 31, 2025

samuelcolvin Mar 31, 2025

samuelcolvin Mar 31, 2025

		"""
		Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning).

Add OpenAI Responses API #1256

Add OpenAI Responses API #1256

Conversation

Kludex commented Mar 27, 2025 • edited Loading

github-actions bot commented Mar 27, 2025 • edited Loading

Docs Preview

samuelcolvin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmontagu Mar 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hyperlint-ai bot commented Mar 28, 2025

PR Change Summary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kludex Mar 31, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kludex commented Mar 27, 2025 •

edited

Loading

github-actions bot commented Mar 27, 2025 •

edited

Loading

dmontagu Mar 28, 2025 •

edited

Loading

Kludex Mar 31, 2025 •

edited

Loading