Skip to content

Conversation

@Milofax
Copy link

@Milofax Milofax commented Jan 15, 2026

Summary

This PR adds support for configuring the small_model parameter in the MCP server's config.yaml file.

Problem

Previously, the small model was hardcoded to either gpt-5-nano or gpt-4.1-mini based on the main model type (see factories.py lines 122-135). This made it impossible to use local LLMs like Ollama which don't have these OpenAI-specific models available.

When using Ollama with Graphiti MCP Server, users would get errors like:

model 'gpt-4.1-mini' not found, try pulling it first

Solution

  • Add optional small_model field to LLMConfig in schema.py
  • Use configured small_model in factories.py if set, otherwise auto-detect as before
  • Bonus fix: Also pass base_url from provider config to CoreLLMConfig (was missing, caused issues with OpenAI-compatible endpoints)

Changes

File Change
mcp_server/src/config/schema.py Add small_model: str | None field
mcp_server/src/services/factories.py Use config.small_model if set + pass base_url
mcp_server/config/*.yaml Add documented small_model option
mcp_server/README.md Update Ollama example with small_model

Example config for Ollama

llm:
  provider: "openai"
  model: "qwen3:32b"
  small_model: "qwen3:32b"  # IMPORTANT for local LLMs
  max_tokens: 4096

  providers:
    openai:
      api_key: "ollama"
      api_url: "http://localhost:11434/v1"

Test plan

  • Verify existing behavior unchanged when small_model not set (auto-detection)
  • Test with Ollama: set small_model to same model as main model
  • Verify base_url is correctly passed to OpenAI client

Fixes #1155

🤖 Generated with Claude Code

This adds support for configuring the `small_model` parameter in the MCP
server's config.yaml. Previously, the small model was hardcoded to either
`gpt-5-nano` or `gpt-4.1-mini` based on the main model type, making it
impossible to use local LLMs like Ollama which don't have these models.

Changes:
- Add `small_model` field to LLMConfig schema (optional, auto-detected if not set)
- Pass configured small_model to CoreLLMConfig in factories.py
- Also pass base_url to CoreLLMConfig for OpenAI provider (was missing)
- Update all example config files with small_model documentation
- Update README with Ollama configuration example highlighting small_model

This fixes getzep#1155 - MCP Server cannot configure small_model for local LLMs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@danielchalef
Copy link
Member

danielchalef commented Jan 15, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@Milofax
Copy link
Author

Milofax commented Jan 16, 2026

I have read the CLA Document and I hereby sign the CLA

@Milofax
Copy link
Author

Milofax commented Jan 16, 2026

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Add small_model configuration to MCP Server config.yaml

2 participants