[Bug]: TypeError when using a model not in model_prices_and_context_window.json

[paul-gauthier]

What happened?

I am seeing the attached exception frequently when using a together.ai model which is not in model_prices_and_context_window.json. The exception seems to be in a background thread and doesn't crash the caller.

I am using together_ai/qwen/Qwen2-72B-Instruct.

Relevant log output

Exception in thread Thread-3 (success_handler):
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/litellm/utils.py", line 1527, in _success_handler_helper_fn
    litellm.response_cost_calculator(
  File "/usr/local/lib/python3.10/site-packages/litellm/cost_calculator.py", line 69, in response_cost_calculator
    response_cost = completion_cost(
  File "/usr/local/lib/python3.10/site-packages/litellm/utils.py", line 4767, in completion_cost
sieve.py
    raise e
  File "/usr/local/lib/python3.10/site-packages/litellm/utils.py", line 4751, in completion_cost
    ) = cost_per_token(
  File "/usr/local/lib/python3.10/site-packages/litellm/utils.py", line 4408, in cost_per_token
    model_with_provider = custom_llm_provider + "/" + model
TypeError: can only concatenate str (not "NoneType") to str

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/site-packages/litellm/utils.py", line 1567, in success_handler
    start_time, end_time, result = self._success_handler_helper_fn(
  File "/usr/local/lib/python3.10/site-packages/litellm/utils.py", line 1561, in _success_handler_helper_fn
    raise Exception(f"[Non-Blocking] LiteLLM.Success_Call Error: {str(e)}")
    Exception: [Non-Blocking] LiteLLM.Success_Call Error: can only concatenate str (not "NoneType") to str

Twitter / LinkedIn details

No response

[krrishdholakia]

one of these is None but i can't tell which one

model_with_provider = custom_llm_provider + "/" + model
TypeError: can only concatenate str (not "NoneType") to str

do you have a sample call i can use to repro this? @paul-gauthier

[paul-gauthier]

model = "together_ai/qwen/Qwen2-72B-Instruct"
messages=[{"role": "user", "content": "What model are you"}]
response = litellm.completion(model=model, messages=messages)