fix(ollama): thread api_base to get_model_info + graceful fallback

[Chesars]

Relevant issues

Fixes #21967
Fixes #9602
Fixes #7997
Fixes #10158

Pre-Submission checklist

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

When users pass api_base to litellm.completion() for Ollama, the LLM call works correctly, but the model info fetch (for context window, function_calling support) ignores the user's api_base and only reads OLLAMA_API_BASE env var or defaults to localhost:11434. This causes confusing error messages in logs and incomplete model metadata.

api_base was never passed through the get_model_info() call chain. OllamaConfig.get_model_info() hardcoded api_base from env var only.

Fix

1. OllamaConfig.get_model_info() — accepts optional api_base, uses it before env var fallback, returns safe defaults on connection failure instead of raising
2. litellm/utils.py — threads api_base through _get_model_info_helper(), _cached_get_model_info_helper(), and get_model_info() (optional param, None default — no impact on other providers)
3. litellm_logging.py — passes api_base from litellm_params through get_model_cost_information() to get_model_info()

Backwards compatibility

• api_base is an optional parameter with default None across all changed signatures
• Non-Ollama providers ignore it entirely — they resolve model info from the static JSON cost map
• @lru_cache behavior unchanged: None is hashable, non-Ollama callers always pass None = same cache key as before
• Ollama with different api_base values = different cache entries (correct: different servers may have different models)

Tests added

4 unit tests in tests/test_litellm/llms/ollama/test_ollama_model_info.py:

• Uses provided api_base over env var
• Falls back to OLLAMA_API_BASE env var when no api_base given
• Returns defaults gracefully on connection error (no exception)
• Strips ollama/ and ollama_chat/ prefixes correctly

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 24, 2026 2:53am

[greptile-apps]

Greptile Summary

This PR fixes a bug where api_base passed to litellm.completion() for Ollama was ignored during model info fetching (get_model_info), causing the metadata lookup to always hit OLLAMA_API_BASE env var or localhost:11434 even when the user specified a different server. The fix also adds graceful fallback — returning safe defaults instead of raising when the Ollama server is unreachable.

• OllamaConfig.get_model_info() now accepts an optional api_base parameter with a fallback chain: provided value → env var → localhost:11434. On connection/HTTP failure, it returns safe defaults instead of raising.
• litellm/utils.py threads api_base through get_model_info(), _cached_get_model_info_helper(), and _get_model_info_helper() — all with Optional[str] = None defaults for backward compatibility.
• litellm_logging.py passes api_base from litellm_params into the model cost information lookup.
• Backward compatibility is maintained: api_base=None default means non-Ollama providers are unaffected, and lru_cache behavior is preserved (None is hashable, same cache key as before for non-Ollama callers).
• 4 unit tests added covering: provided api_base usage, env var fallback, graceful error handling, and prefix stripping. All tests use mocks.

Confidence Score: 4/5

• This PR is safe to merge — changes are backward-compatible, well-scoped, and covered by tests.
• The fix correctly threads api_base through the model info call chain with optional parameters and None defaults, ensuring no impact on non-Ollama providers. The lru_cache behavior is preserved. The graceful fallback is a reasonable improvement over raising on connection errors. One minor concern: debug-level logging on errors may make it hard for users to notice issues like misspelled model names, but this is a style preference rather than a correctness issue.
• litellm/llms/ollama/completion/transformation.py — the error logging level (debug vs warning) may be too quiet for production troubleshooting.

Important Files Changed

Filename	Overview
litellm/llms/ollama/completion/transformation.py	Core fix: get_model_info() now accepts optional api_base parameter with fallback chain (param -> env var -> default). Error handling changed from raising to returning safe defaults. Minor concern: debug-level logging may hide real errors like misspelled model names.
litellm/utils.py	Threads api_base through get_model_info(), _get_model_info_helper(), and _cached_get_model_info_helper(). All are optional with None default. Removes now-unnecessary OllamaError re-raise. lru_cache behavior preserved correctly (None is hashable, different api_base values create different cache entries).
litellm/litellm_core_utils/litellm_logging.py	Passes api_base from litellm_params through get_model_cost_information() to get_model_info(). Clean, minimal changes with proper optional parameter handling.
tests/test_litellm/llms/ollama/test_ollama_model_info.py	Four well-structured unit tests covering: provided api_base usage, env var fallback, graceful connection error handling, and prefix stripping. All tests use mocks (no real network calls).

Sequence Diagram

sequenceDiagram
    participant User as litellm.completion()
    participant Logging as litellm_logging.py
    participant Utils as get_model_info()
    participant Helper as _get_model_info_helper()
    participant Ollama as OllamaConfig.get_model_info()
    participant Server as Ollama Server

    User->>Logging: api_base in litellm_params
    Logging->>Utils: get_model_info(model, provider, api_base)
    Utils->>Helper: _get_model_info_helper(model, provider, api_base)
    Helper->>Ollama: get_model_info(model, api_base)
    
    alt api_base provided
        Ollama->>Server: POST api_base/api/show
    else env var OLLAMA_API_BASE set
        Ollama->>Server: POST env_var/api/show
    else default
        Ollama->>Server: POST localhost:11434/api/show
    end

    alt Success
        Server-->>Ollama: model metadata
        Ollama-->>Helper: ModelInfoBase with actual values
    else Connection or HTTP Error
        Server--xOllama: Error
        Ollama-->>Helper: ModelInfoBase with safe defaults
    end

    Helper-->>Utils: ModelInfoBase
    Utils-->>Logging: ModelInfo

Loading

_{Last reviewed commit: 1067705}

[greptile-apps]

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}