ollama APIConnectionError

[a7mad911]

error when using ollama

from agents import Agent, Runner, AsyncOpenAI, ModelSettings, OpenAIChatCompletionsModel, set_default_openai_client
from openai import AsyncOpenAI
external_client = AsyncOpenAI(
api_key="",
base_url="http://localhost:11434/v1",
)

set_default_openai_client(external_client)

spanish_agent = Agent(
name="Spanish agent",
instructions="You only speak English.",
model=OpenAIChatCompletionsModel(
model="llama3.1",
openai_client=external_client,
),
model_settings=ModelSettings(temperature=0.5),
)
res = Runner.run_sync(
starting_agent=spanish_agent,
input="hi"
)

print(res.final_output)

error:
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
OPENAI_API_KEY is not set, skipping trace export

[LeonG7]

#44

[keldenl]

I tried #44 and it didn't work out of the box. Had to do a mix of what was on that thread and https://openai.github.io/openai-agents-python/models/ but it works with ollama!

This is an example with the starter code:

from openai import AsyncOpenAI
from agents import OpenAIChatCompletionsModel,Agent,Runner
from agents.model_settings import ModelSettings
from agents import set_default_openai_client, set_tracing_disabled

external_client = AsyncOpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)
set_default_openai_client(external_client)
set_tracing_disabled(True)


agent = Agent(
    name="Assistant", 
    instructions="You are a helpful assistant",
    model=OpenAIChatCompletionsModel(
        model="gemma3:12b",
        openai_client=external_client,
    )
    )

result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
print(result.final_output)

# Code within the code,
# Functions calling themselves,
# Infinite loop's dance.

[keldenl]

Another issue I ran into was that models that DO NOT support tool calling will NOT work with the handoff example. Any ollama models that support tool calling (https://ollama.com/search?c=tools), will work.

Here's a working example with qwen 2.5 with the example agent triage handoff:

from openai import AsyncOpenAI
from agents import OpenAIChatCompletionsModel,Agent,Runner
from agents.model_settings import ModelSettings
from agents import set_default_openai_client, set_tracing_disabled

external_client = AsyncOpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)
set_default_openai_client(external_client)
set_tracing_disabled(True)


from agents import Agent, Runner
import asyncio

model = OpenAIChatCompletionsModel(
        model="qwen2.5:14b-instruct-q5_K_M",
        openai_client=external_client,
    )

spanish_agent = Agent(
    name="Spanish agent",
    instructions="You only speak Spanish.",
    model=model
)

english_agent = Agent(
    name="English agent",
    instructions="You only speak English",
    model=model
)

triage_agent = Agent(
    name="Triage agent",
    instructions="Handoff to the appropriate agent based on the language of the request.",
    handoffs=[spanish_agent, english_agent],
    model=model
)


async def main():
    result = await Runner.run(triage_agent, input="Hola, ¿cómo estás?")
    print(result.final_output)
    # ¡Hola! Estoy bien, gracias por preguntar. ¿Y tú, cómo estás?


if __name__ == "__main__":
    asyncio.run(main())