fix(exceptions): don't require a provider's optional SDK to normalize errors - #1898
Merged
RobinPicard merged 1 commit intoJul 3, 2026
Merged
Conversation
RobinPicard
approved these changes
Jul 2, 2026
RobinPicard
left a comment
Contributor
There was a problem hiding this comment.
Very thorough testing, thank you!
… errors `is_provider_exception` and `normalize_provider_exception` are public API (both exported in `__all__`). Their documented contract is to let non-provider errors pass through untouched and otherwise fall back to HTTP status-code inspection. `_build_exception_map` broke that contract: for every provider except `mistral` it did a bare `import openai` / `import anthropic` / etc. When the named provider's optional SDK was not installed, building the map raised `ModuleNotFoundError` instead of degrading gracefully. So calling `is_provider_exception(TypeError(...), "openai")` without the `openai` package installed crashed with an import error that masks the user's original exception. This is reachable whenever a caller uses the public helpers for a provider whose SDK is not the one installed (a common multi-provider / minimal-install setup), and it fires inside the `normalize_provider_errors` context manager too. The `mistral` branch already guarded its optional import with `try/except ImportError`; this makes the other six providers consistent. Fix: split the SDK-dependent body into `_provider_exception_map` and wrap the single call site in `try/except ImportError`, returning an empty map when the SDK is absent. Callers then fall through to the SDK-free status-code inspection, which is unchanged. Behavior when the SDK *is* installed is identical. Evidence: 13 existing tests in `test_exceptions.py` (`TestIsProviderException`, `TestNormalizeProviderErrors`) already fail when `openai` is not installed, because they deliberately test with synthetic errors and do not `importorskip` (unlike the sibling `TestExceptionMap*` classes, which correctly do). Those 13 now pass. Added `TestProviderSDKNotInstalled`: parametrized over all providers, it blocks each optional import via `sys.modules[name] = None` so the missing -SDK path is exercised deterministically even in CI where every SDK is installed. The new tests fail without this fix and pass with it.
RobinPicard
force-pushed
the
fix/exceptions-optional-sdk-import
branch
from
July 2, 2026 14:40
321c262 to
e968a6e
Compare
|
📚 Documentation preview: https://dottxt-ai.github.io/outlines/pr-preview/pr-1898/ Preview updates automatically with each commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
is_provider_exceptionandnormalize_provider_exceptionare public API (both inexceptions.__all__). Their documented contract is to let non-provider errors pass through untouched and otherwise fall back to HTTP status-code inspection._build_exception_mapbroke that contract. For every provider exceptmistralit performed a bare optional-SDK import (import openai,import anthropic,from google.genai import errors,import ollama,import huggingface_hub.errors,import urllib3.exceptions). When the named provider's optional SDK is not installed, building the map raisesModuleNotFoundErrorinstead of degrading gracefully — so error-handling code introduces a new failure that masks the user's original exception.This also fires inside the
normalize_provider_errors(provider)context manager (used throughoutoutlines/models/*.py): a plainTypeErrorraised in that block surfaces asModuleNotFoundErrorrather than propagating unchanged.The
mistralbranch already guards its optional import withtry/except ImportError. This PR makes the other six providers consistent with that existing pattern.Fix
Split the SDK-dependent body into
_provider_exception_mapand wrap the single call site intry/except ImportError, returning an empty map when the SDK is absent. Callers then fall through to the SDK-free status-code inspection, which is unchanged. Behavior when the SDK is installed is byte-for-byte identical (same dict, same lookups).Evidence
13 existing tests already fail without this fix when
openaiis not installed —TestIsProviderExceptionandTestNormalizeProviderErrorsdeliberately test with synthetic errors and do notimportorskip, unlike the siblingTestExceptionMap*classes which correctly guard on the SDK. That asymmetry encodes the intended contract; the bug violates it.New regression test
TestProviderSDKNotInstalledblocks each provider's optional import viasys.modules[name] = None, so the missing-SDK path is exercised deterministically even in CI where every SDK is installed. Parametrized across all providers, it asserts:_build_exception_map(provider)returns{}instead of raisingis_provider_exception(TypeError(...), provider)returnsFalse(contract: pass programmer errors through)429 -> RateLimitErrorwith the SDK absentnormalize_provider_errors(provider)re-raises aTypeErrorunchangedProven to guard the bug (stash the source change, tests fail; restore, tests pass):
Full
tests/test_exceptions.pywith all SDKs present: 117 passed, 47 skipped. Broader pure-Python suites (tests/types,tests/test_utils,tests/test_templates.py,tests/test_inputs.py): 249 passed, no regressions.ruff check --config=pyproject.tomlclean on both files;mypyintroduces no new errors (the two pre-existingurllib3stub warnings are unrelated and identical onmain).Changes
src/outlines/exceptions.py— guard the optional-SDK import once; add_provider_exception_maphelper (+13/−0)tests/test_exceptions.py— addTestProviderSDKNotInstalledregression coverage (+65/−0)