Added examples and documentation for LiteLLM model provider & exposed LiteLLMProvider

pontus-devoteam · pontus-devoteam · commit ce1d5f1306da · 2025-03-13T15:34:16.000+01:00
diff --git a/docs/models.md b/docs/models.md
@@ -71,3 +71,155 @@ spanish_agent = Agent(
     model_settings=ModelSettings(temperature=0.5),
 )
 ```
+
+## Using LiteLLM Provider
+
+The SDK includes built-in support for [LiteLLM](https://docs.litellm.ai/), a unified interface for multiple LLM providers. LiteLLM provides a proxy server that exposes an OpenAI-compatible API for various LLM providers including OpenAI, Anthropic, Azure, AWS Bedrock, Google, and more.
+
+### Basic Usage
+
+```python
+from agents import Agent, Runner, LiteLLMProvider
+import asyncio
+
+# Create a LiteLLM provider
+provider = LiteLLMProvider(
+    api_key="your-litellm-api-key",  # or set LITELLM_API_KEY env var
+    base_url="http://localhost:8000", # or set LITELLM_API_BASE env var
+)
+
+# Create an agent using a specific model
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant.",
+    model="claude-3",  # Will be routed to Anthropic
+    model_provider=provider,
+)
+
+async def main():
+    result = await Runner.run(agent, input="Hello!")
+    print(result.final_output)
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+### Environment Variables
+
+The LiteLLM provider supports configuration through environment variables:
+
+```bash
+# LiteLLM configuration
+export LITELLM_API_KEY="your-litellm-api-key"
+export LITELLM_API_BASE="http://localhost:8000"
+export LITELLM_MODEL="gpt-4"  # Default model (optional)
+
+# Provider-specific keys (examples)
+export OPENAI_API_KEY="sk-..."
+export ANTHROPIC_API_KEY="sk-ant-..."
+export AZURE_API_KEY="..."
+export AWS_ACCESS_KEY_ID="..."
+export AWS_SECRET_ACCESS_KEY="..."
+```
+
+### Model Routing
+
+The provider automatically routes model names to their appropriate providers:
+
+```python
+# Models are automatically routed based on their names
+openai_agent = Agent(
+    name="OpenAI Agent",
+    instructions="Using GPT-4",
+    model="gpt-4",  # Will be routed to OpenAI
+    model_provider=provider,
+)
+
+anthropic_agent = Agent(
+    name="Anthropic Agent",
+    instructions="Using Claude",
+    model="claude-3",  # Will be routed to Anthropic
+    model_provider=provider,
+)
+
+azure_agent = Agent(
+    name="Azure Agent",
+    instructions="Using Azure OpenAI",
+    model="azure/gpt-4",  # Explicitly using Azure
+    model_provider=provider,
+)
+```
+
+You can also explicitly specify providers using prefixes:
+
+- `openai/` - OpenAI models
+- `anthropic/` - Anthropic models
+- `azure/` - Azure OpenAI models
+- `aws/` - AWS Bedrock models
+- `cohere/` - Cohere models
+- `replicate/` - Replicate models
+- `huggingface/` - Hugging Face models
+- `mistral/` - Mistral AI models
+- `gemini/` - Google Gemini models
+- `groq/` - Groq models
+
+### Advanced Configuration
+
+The provider supports additional configuration options:
+
+```python
+provider = LiteLLMProvider(
+    api_key="your-litellm-api-key",
+    base_url="http://localhost:8000",
+    model_name="gpt-4",  # Default model
+    use_responses=True,  # Use OpenAI Responses API format
+    extra_headers={      # Additional headers
+        "x-custom-header": "value"
+    },
+    drop_params=True,    # Drop unsupported params for specific models
+)
+```
+
+### Using Multiple Providers
+
+You can use different providers for different agents in your workflow:
+
+```python
+from agents import Agent, Runner, OpenAIProvider, LiteLLMProvider
+import asyncio
+
+# OpenAI provider for direct OpenAI API access
+openai_provider = OpenAIProvider()
+
+# LiteLLM provider for other models
+litellm_provider = LiteLLMProvider(
+    api_key="your-litellm-api-key",
+    base_url="http://localhost:8000"
+)
+
+# Agent using OpenAI directly
+triage_agent = Agent(
+    name="Triage",
+    instructions="Route requests to appropriate agents",
+    model="gpt-3.5-turbo",
+    model_provider=openai_provider,
+)
+
+# Agent using Claude through LiteLLM
+analysis_agent = Agent(
+    name="Analysis",
+    instructions="Perform detailed analysis",
+    model="claude-3",
+    model_provider=litellm_provider,
+)
+
+async def main():
+    result = await Runner.run(
+        triage_agent, 
+        input="Analyze this data",
+        handoffs=[analysis_agent]
+    )
+    print(result.final_output)
+```
+
+The LiteLLM provider makes it easy to use multiple LLM providers while maintaining a consistent interface and the full feature set of the Agents SDK including handoffs, tools, and tracing.
diff --git a/examples/litellm/README.md b/examples/litellm/README.md
@@ -0,0 +1,52 @@
+# LiteLLM Provider Examples
+
+This directory contains examples demonstrating how to use the LiteLLM provider with the Agents SDK.
+
+## Prerequisites
+
+1. Install and run the LiteLLM proxy server:
+```bash
+pip install litellm
+litellm --model ollama/llama2 --port 8000
+```
+
+2. Set up environment variables:
+```bash
+# LiteLLM configuration
+export LITELLM_API_KEY="your-litellm-api-key"  # If required by your proxy
+export LITELLM_API_BASE="http://localhost:8000"
+
+# Provider API keys (as needed)
+export OPENAI_API_KEY="sk-..."
+export ANTHROPIC_API_KEY="sk-ant-..."
+export GEMINI_API_KEY="..."
+```
+
+## Examples
+
+### Multi-Provider Workflow (`multi_provider_workflow.py`)
+
+This example demonstrates using multiple LLM providers in a workflow:
+
+1. A triage agent (using OpenAI directly) determines the task type
+2. Based on the task type, it routes to specialized agents:
+   - Summarization tasks → Claude (via LiteLLM)
+   - Coding tasks → GPT-4 (via LiteLLM)
+   - Creative tasks → Gemini (via LiteLLM)
+
+To run:
+```bash
+python examples/litellm/multi_provider_workflow.py
+```
+
+The example will process three different types of requests to demonstrate the routing:
+1. A summarization request about the French Revolution
+2. A coding request to implement a Fibonacci sequence
+3. A creative writing request about a time-traveling coffee cup
+
+## Notes
+
+- The LiteLLM provider automatically routes model names to their appropriate providers (e.g., `claude-3` → Anthropic, `gpt-4` → OpenAI)
+- You can explicitly specify providers using prefixes (e.g., `anthropic/claude-3`, `openai/gpt-4`)
+- The provider handles passing API keys and configuration through headers
+- All Agents SDK features (handoffs, tools, tracing) work with the LiteLLM provider 
diff --git a/examples/litellm/multi_provider_workflow.py b/examples/litellm/multi_provider_workflow.py
@@ -0,0 +1,133 @@
+"""
+This example demonstrates using multiple LLM providers in a workflow using LiteLLM.
+It creates a workflow where:
+1. A triage agent (using OpenAI directly) determines the task type
+2. Based on the task type, it routes to:
+   - A summarization agent using Claude via LiteLLM
+   - A coding agent using GPT-4 via LiteLLM
+   - A creative agent using Gemini via LiteLLM
+"""
+
+import asyncio
+import os
+from typing import Literal
+
+from agents import Agent, Runner, OpenAIProvider, LiteLLMProvider
+from agents.agent_output import AgentOutputSchema
+from pydantic import BaseModel
+
+
+class TaskType(BaseModel):
+    """The type of task to be performed."""
+    task: Literal["summarize", "code", "creative"]
+    explanation: str
+
+
+class TaskOutput(BaseModel):
+    """The output of the task."""
+    result: str
+    provider_used: str
+
+
+# Set up providers
+openai_provider = OpenAIProvider(
+    api_key=os.getenv("OPENAI_API_KEY")
+)
+
+litellm_provider = LiteLLMProvider(
+    api_key=os.getenv("LITELLM_API_KEY"),
+    base_url=os.getenv("LITELLM_API_BASE", "http://localhost:8000")
+)
+
+# Create specialized agents for different tasks
+triage_agent = Agent(
+    name="Triage Agent",
+    instructions="""
+    You are a triage agent that determines the type of task needed.
+    - For text analysis, summarization, or understanding tasks, choose 'summarize'
+    - For programming, coding, or technical tasks, choose 'code'
+    - For creative writing, storytelling, or artistic tasks, choose 'creative'
+    """,
+    model="gpt-3.5-turbo",
+    model_provider=openai_provider,
+    output_schema=AgentOutputSchema(TaskType),
+)
+
+summarize_agent = Agent(
+    name="Summarization Agent",
+    instructions="""
+    You are a summarization expert using Claude's advanced comprehension capabilities.
+    Provide clear, concise summaries while preserving key information.
+    """,
+    model="claude-3",  # Will be routed to Anthropic
+    model_provider=litellm_provider,
+    output_schema=AgentOutputSchema(TaskOutput),
+)
+
+code_agent = Agent(
+    name="Coding Agent",
+    instructions="""
+    You are a coding expert using GPT-4's technical capabilities.
+    Provide clean, well-documented code solutions.
+    """,
+    model="gpt-4",  # Will be routed to OpenAI
+    model_provider=litellm_provider,
+    output_schema=AgentOutputSchema(TaskOutput),
+)
+
+creative_agent = Agent(
+    name="Creative Agent",
+    instructions="""
+    You are a creative writer using Gemini's creative capabilities.
+    Create engaging, imaginative content.
+    """,
+    model="gemini-pro",  # Will be routed to Google
+    model_provider=litellm_provider,
+    output_schema=AgentOutputSchema(TaskOutput),
+)
+
+
+async def process_request(user_input: str) -> str:
+    """Process a user request using the appropriate agent."""
+    
+    # First, triage the request
+    triage_result = await Runner.run(
+        triage_agent,
+        input=f"What type of task is this request? {user_input}"
+    )
+    task_type = triage_result.output
+    
+    # Route to the appropriate agent
+    target_agent = {
+        "summarize": summarize_agent,
+        "code": code_agent,
+        "creative": creative_agent,
+    }[task_type.task]
+    
+    # Process with the specialized agent
+    result = await Runner.run(target_agent, input=user_input)
+    return f"""
+Task Type: {task_type.task}
+Reason: {task_type.explanation}
+Result: {result.output.result}
+Provider Used: {result.output.provider_used}
+"""
+
+
+async def main():
+    """Run example requests through the workflow."""
+    requests = [
+        "Can you summarize the key points of the French Revolution?",
+        "Write a Python function to calculate the Fibonacci sequence.",
+        "Write a short story about a time-traveling coffee cup.",
+    ]
+    
+    for request in requests:
+        print(f"\nProcessing request: {request}")
+        print("-" * 80)
+        result = await process_request(request)
+        print(result)
+
+
+if __name__ == "__main__":
+    asyncio.run(main()) 
diff --git a/src/agents/__init__.py b/src/agents/__init__.py
@@ -44,6 +44,7 @@
 from .models.openai_chatcompletions import OpenAIChatCompletionsModel
 from .models.openai_provider import OpenAIProvider
 from .models.openai_responses import OpenAIResponsesModel
+from .models.litellm_provider import LiteLLMProvider
 from .result import RunResult, RunResultStreaming
 from .run import RunConfig, Runner
 from .run_context import RunContextWrapper, TContext
@@ -139,6 +140,7 @@ def enable_verbose_stdout_logging():
     "OpenAIChatCompletionsModel",
     "OpenAIProvider",
     "OpenAIResponsesModel",
+    "LiteLLMProvider",
     "AgentOutputSchema",
     "Computer",
     "AsyncComputer",
