Seems like LiteLLM (docker) insists on Azure?

[CaptainPalapa]

I don't know if it's me. I'm not a Docker expert, but....
Here's what the logs keep outputting and, of course, the container just keeps restarting over and over until I docker compose down.

[chopped out]
Loaded config YAML (api_key and environment_variables are not shown):
  "model_list": [
    {
      "model_name": "gpt-4",
      "litellm_params": {
        "model": "azure/chatgpt-v-2",
        "api_base": "https://openai-gpt-4-test-v-1.openai.azure.com/",
        "api_version": "2023-05-15"
      }
    },
    {
      "model_name": "gpt-4",
      "litellm_params": {
        "api_base": "https://openai-gpt-4-test-v-2.openai.azure.com/",
        "rpm": 100
      }
    },
    {
      "model_name": "gpt-4",
      "litellm_params": {
        "model": "azure/gpt-4",
        "api_base": "https://openai-gpt-4-test-v-2.openai.azure.com/",
        "rpm": 10
INFO:     Waiting for application startup.
ERROR:    Traceback (most recent call last):
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 566, in __aenter__
    await self._router.startup()
    await handler()
  File "/usr/local/lib/python3.9/site-packages/litellm/proxy/proxy_server.py", line 750, in startup_event
    initialize(**worker_config)
  File "/usr/local/lib/python3.9/site-packages/litellm/proxy/proxy_server.py", line 637, in initialize
    llm_router, llm_model_list, general_settings = load_router_config(router=llm_router, config_file_path=config)
    router = litellm.Router(model_list=model_list, num_retries=3)
  File "/usr/local/lib/python3.9/site-packages/litellm/router.py", line 78, in __init__
    self.set_model_list(model_list)
  File "/usr/local/lib/python3.9/site-packages/litellm/router.py", line 880, in set_model_list
    model["async_client"] = openai.AsyncAzureOpenAI(
  File "/usr/local/lib/python3.9/site-packages/openai/lib/azure.py", line 399, in __init__
    raise OpenAIError(

ERROR:    Application startup failed. Exiting.
    }
  "litellm_settings": {
    "drop_params": true,
    "set_verbose": true
  },
  "general_settings": null
}
LiteLLM.Router: Initializing OpenAI Client for azure/chatgpt-v-2, https://openai-gpt-4-test-v-1.openai.azure.com/

I have ZERO idea where this Azure stuff is coming from. I've verified that my config is getting loaded, by attaching a terminal and doing a cat on the config file. So my docker compose env / pathing has got to be right. I've also verified that environment variables are making it in, too. I'm attaching my compose and config files. You can see in the attached config that there is nothing but ollama at the moment.

If anyone has ANY idea, please let me know. I'm a 25 year veteran programmer, so ... I'm sharp enough, but this one is eluding me after almost 2 days. I've even thrown the AI's at it directly (cursor, also the web-based), and I can't solve it. (Okay, .yaml files no good, so ... they're in a .zip.)

ai-stack-compose-and-config.zip

[CaptainPalapa]

So, I ended up using the compose file from the repo. (I'm NOT checking out a whole repo!) And I did get it running. Still not sure why this all happened, but I WAS able to move past it. Kudos to a great concept!