Incompatibility between openai and fastapi

[LarryMartell]

Anyone using fastapi and openai-python? I am trying to, but running into a pydantic issue. Using python '3.12.3', fastapi '0.111.0', openai '1.30.4' (although the same happens with python 3.8 and 3.10).

(3.12venv) Delphi:dispatcher LarryMartell$ python
Python 3.12.3 (v3.12.3:f6650f9ad7, Apr  9 2024, 08:18:48) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from fastapi import FastAPI
>>> from openai.types.chat import CompletionCreateParams
>>> app = FastAPI()
>>> @app.post("/v1/chat/completions")
... def chat_completions(r: CompletionCreateParams):
...  pass
... 
Traceback (most recent call last):
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 210, in __init__
    core_schema = _getattr_no_parents(type, '__pydantic_core_schema__')
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 98, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/routing.py", line 944, in decorator
    self.add_api_route(
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/routing.py", line 883, in add_api_route
    route = route_class(
            ^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/routing.py", line 513, in __init__
    self.dependant = get_dependant(path=self.path_format, call=self.endpoint)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/dependencies/utils.py", line 261, in get_dependant
    type_annotation, depends, param_field = analyze_param(
                                            ^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/dependencies/utils.py", line 444, in analyze_param
    field = create_response_field(
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/utils.py", line 99, in create_response_field
    return ModelField(**kwargs)  # type: ignore[arg-type]
           ^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 6, in __init__
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/fastapi/_compat.py", line 109, in __post_init__
    self._type_adapter: TypeAdapter[Any] = TypeAdapter(
                                           ^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 212, in __init__
    core_schema = _get_schema(type, config_wrapper, parent_depth=_parent_depth + 1)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 81, in _get_schema
    schema = gen.generate_schema(type_)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 502, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 737, in _generate_schema_inner
    return self._annotated_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1757, in _annotated_schema
    schema = self._apply_annotations(source_type, annotations)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1825, in _apply_annotations
    schema = get_inner_schema(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 82, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1907, in new_handler
    schema = metadata_get_schema(source, get_inner_schema)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1903, in <lambda>
    lambda source, handler: handler(source)
                            ^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 82, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1806, in inner_handler
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 758, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 840, in match_type
    return self._match_generic_type(obj, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 864, in _match_generic_type
    return self._union_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1152, in _union_schema
    choices.append(self.generate_schema(arg))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 502, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 758, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 806, in match_type
    return self._typed_dict_schema(obj, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1254, in _typed_dict_schema
    for field_name, annotation in get_type_hints_infer_globalns(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/LarryMartell/dispatcher/3.12venv/lib/python3.12/site-packages/pydantic/_internal/_fields.py", line 56, in get_type_hints_infer_globalns
    return get_type_hints(obj, globalns=globalns, localns=localns, include_extras=include_extras)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 2244, in get_type_hints
    value = _eval_type(value, base_globals, base_locals)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 414, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 924, in _evaluate
    eval(self.__forward_code__, globalns, localns),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
TypeError: 'pydantic_core._pydantic_core.PydanticUndefinedType' object is not subscriptable

[CharlyJazz]

Just use custom model create by you or something like that. chill

[LarryMartell]

Yes, but I wanted to use the offical PyPI package.

[CharlyJazz]

And also you should not expose internal system details in a query param like that. You can open a door to vulnerabilities

[LarryMartell]

yes, this was just an example.