Skip to content

fix(exceptions): don't require a provider's optional SDK to normalize errors - #1898

Merged
RobinPicard merged 1 commit into
dottxt-ai:mainfrom
Sanjays2402:fix/exceptions-optional-sdk-import
Jul 3, 2026
Merged

fix(exceptions): don't require a provider's optional SDK to normalize errors#1898
RobinPicard merged 1 commit into
dottxt-ai:mainfrom
Sanjays2402:fix/exceptions-optional-sdk-import

Conversation

@Sanjays2402

Copy link
Copy Markdown
Contributor

Summary

is_provider_exception and normalize_provider_exception are public API (both in exceptions.__all__). Their documented contract is to let non-provider errors pass through untouched and otherwise fall back to HTTP status-code inspection.

_build_exception_map broke that contract. For every provider except mistral it performed a bare optional-SDK import (import openai, import anthropic, from google.genai import errors, import ollama, import huggingface_hub.errors, import urllib3.exceptions). When the named provider's optional SDK is not installed, building the map raises ModuleNotFoundError instead of degrading gracefully — so error-handling code introduces a new failure that masks the user's original exception.

# outlines installed, openai SDK not installed:
from outlines.exceptions import is_provider_exception
is_provider_exception(TypeError("a bug in my code"), "openai")
# ModuleNotFoundError: No module named 'openai'   <- should return False

This also fires inside the normalize_provider_errors(provider) context manager (used throughout outlines/models/*.py): a plain TypeError raised in that block surfaces as ModuleNotFoundError rather than propagating unchanged.

The mistral branch already guards its optional import with try/except ImportError. This PR makes the other six providers consistent with that existing pattern.

Fix

Split the SDK-dependent body into _provider_exception_map and wrap the single call site in try/except ImportError, returning an empty map when the SDK is absent. Callers then fall through to the SDK-free status-code inspection, which is unchanged. Behavior when the SDK is installed is byte-for-byte identical (same dict, same lookups).

@lru_cache(maxsize=16)
def _build_exception_map(provider: str) -> dict[type, type[APIError]]:
    try:
        return _provider_exception_map(provider)
    except ImportError:
        return {}

Evidence

13 existing tests already fail without this fix when openai is not installed — TestIsProviderException and TestNormalizeProviderErrors deliberately test with synthetic errors and do not importorskip, unlike the sibling TestExceptionMap* classes which correctly guard on the SDK. That asymmetry encodes the intended contract; the bug violates it.

# openai not installed, before fix:
13 failed, 59 passed, 67 skipped
# openai not installed, after fix:
72 passed, 67 skipped

New regression test TestProviderSDKNotInstalled blocks each provider's optional import via sys.modules[name] = None, so the missing-SDK path is exercised deterministically even in CI where every SDK is installed. Parametrized across all providers, it asserts:

  • _build_exception_map(provider) returns {} instead of raising
  • is_provider_exception(TypeError(...), provider) returns False (contract: pass programmer errors through)
  • status-code fallback still maps 429 -> RateLimitError with the SDK absent
  • normalize_provider_errors(provider) re-raises a TypeError unchanged

Proven to guard the bug (stash the source change, tests fail; restore, tests pass):

# new tests, source fix stashed:  25 failed
# new tests, source fix applied:  25 passed

Full tests/test_exceptions.py with all SDKs present: 117 passed, 47 skipped. Broader pure-Python suites (tests/types, tests/test_utils, tests/test_templates.py, tests/test_inputs.py): 249 passed, no regressions. ruff check --config=pyproject.toml clean on both files; mypy introduces no new errors (the two pre-existing urllib3 stub warnings are unrelated and identical on main).

Changes

  • src/outlines/exceptions.py — guard the optional-SDK import once; add _provider_exception_map helper (+13/−0)
  • tests/test_exceptions.py — add TestProviderSDKNotInstalled regression coverage (+65/−0)

@RobinPicard RobinPicard left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very thorough testing, thank you!

… errors

`is_provider_exception` and `normalize_provider_exception` are public API
(both exported in `__all__`). Their documented contract is to let
non-provider errors pass through untouched and otherwise fall back to
HTTP status-code inspection.

`_build_exception_map` broke that contract: for every provider except
`mistral` it did a bare `import openai` / `import anthropic` / etc. When
the named provider's optional SDK was not installed, building the map
raised `ModuleNotFoundError` instead of degrading gracefully. So calling
`is_provider_exception(TypeError(...), "openai")` without the `openai`
package installed crashed with an import error that masks the user's
original exception. This is reachable whenever a caller uses the public
helpers for a provider whose SDK is not the one installed (a common
multi-provider / minimal-install setup), and it fires inside the
`normalize_provider_errors` context manager too.

The `mistral` branch already guarded its optional import with
`try/except ImportError`; this makes the other six providers consistent.

Fix: split the SDK-dependent body into `_provider_exception_map` and wrap
the single call site in `try/except ImportError`, returning an empty map
when the SDK is absent. Callers then fall through to the SDK-free
status-code inspection, which is unchanged. Behavior when the SDK *is*
installed is identical.

Evidence: 13 existing tests in `test_exceptions.py`
(`TestIsProviderException`, `TestNormalizeProviderErrors`) already fail
when `openai` is not installed, because they deliberately test with
synthetic errors and do not `importorskip` (unlike the sibling
`TestExceptionMap*` classes, which correctly do). Those 13 now pass.

Added `TestProviderSDKNotInstalled`: parametrized over all providers, it
blocks each optional import via `sys.modules[name] = None` so the missing
-SDK path is exercised deterministically even in CI where every SDK is
installed. The new tests fail without this fix and pass with it.
@RobinPicard
RobinPicard force-pushed the fix/exceptions-optional-sdk-import branch from 321c262 to e968a6e Compare July 2, 2026 14:40
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

📚 Documentation preview: https://dottxt-ai.github.io/outlines/pr-preview/pr-1898/

Preview updates automatically with each commit.

@RobinPicard
RobinPicard merged commit 5527d3e into dottxt-ai:main Jul 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants