📖 docs: Add custom models new ENV set and ollama non-local configurat…

…ions (lobehub#1969) * 📖 docs: Add custom models new ENV set and ollama non-local configurations. * 📖 docs: fix more CUSTOM_Models in docs * Update .env.example * Update azure-openai.mdx * Update azure-openai.zh-CN.mdx
jjcruise · Apr 11, 2024 · 9babeb9 · 9babeb9
1 parent 5ab9afb
commit 9babeb9
Show file tree

Hide file tree

Showing 14 changed files with 120 additions and 120 deletions.
diff --git a/.env.example b/.env.example
@@ -1,114 +1,105 @@
 # add a access code to lock your lobe-chat application, you can set a long password to avoid leaking. If this value contains a comma, it is a password array.
-#ACCESS_CODE=lobe66
+# ACCESS_CODE=lobe66
 
 # Specify your API Key selection method, currently supporting `random` and `turn`.
 # API_KEY_SELECT_MODE=random
 
 
 ########################################
-############ OpenAI Service ############
+######## Model Provider Service ########
 ########################################
 
+### OpenAI ###
+
 # you openai api key
 OPENAI_API_KEY=sk-xxxxxxxxx
 
 # use a proxy to connect to the OpenAI API
 # OPENAI_PROXY_URL=https://api.openai.com/v1
 
 # add your custom model name, multi model separate by comma. for example gpt-3.5-1106,gpt-4-1106
-#OPENAI_ENABLED_MODELS=gpt-3.5-turbo
+# OPENAI_MODEL_LIST=gpt-3.5-turbo
 
-########################################
-######### Azure OpenAI Service #########
-########################################
-# you can learn azure OpenAI Service on https://learn.microsoft.com/en-us/azure/ai-services/openai/overview
 
+### Azure OpenAI ###
+
+# you can learn azure OpenAI Service on https://learn.microsoft.com/en-us/azure/ai-services/openai/overview
 # use Azure OpenAI Service by uncomment the following line
 
 # The API key you applied for on the Azure OpenAI account page, which can be found in the "Keys and Endpoints" section.
 # AZURE_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
 # The endpoint you applied for on the Azure OpenAI account page, which can be found in the "Keys and Endpoints" section.
-# OPENAI_PROXY_URL=https://docs-test-001.openai.azure.com
+# AZURE_ENDPOINT=https://docs-test-001.openai.azure.com
 
 # Azure's API version, follows the YYYY-MM-DD format
-# AZURE_API_VERSION=2023-08-01-preview
+# AZURE_API_VERSION=2024-02-01
 
-########################################
-############ ZhiPu AI Service ##########
-########################################
 
-#ZHIPU_API_KEY=xxxxxxxxxxxxxxxxxxx.xxxxxxxxxxxxx
+### Anthropic Service ####
 
-########################################
-########## Moonshot AI Service #########
-########################################
+# ANTHROPIC_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-#MOONSHOT_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+# use a proxy to connect to the Anthropic API
+# ANTHROPIC_PROXY_URL=https://api.anthropic.com
 
-########################################
-########### Google AI Service ##########
-########################################
 
-#GOOGLE_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+### Google AI  ####
 
-########################################
-######### AWS Bedrock Service ##########
-########################################
+# GOOGLE_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-#AWS_REGION=us-east-1
-#AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxx
-#AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-########################################
-######### Ollama AI Service ##########
-########################################
+### AWS Bedrock  ###
+
+# AWS_REGION=us-east-1
+# AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxx
+# AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+
+### Ollama AI ####
 
 # You can use ollama to get and run LLM locally, learn more about it via https://github.com/ollama/ollama
+
 # The local/remote ollama service url
 # OLLAMA_PROXY_URL=http://127.0.0.1:11434/v1
 
-########### Mistral AI Service ##########
-########################################
+# OLLAMA_MODEL_LIST=your_ollama_model_names
+
+
+### OpenRouter Service ###
+
+# OPENROUTER_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+# OPENROUTER_MODEL_LIST=model1,model2,model3
+
+
+### Mistral AI ###
 
 # MISTRAL_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-########################################
-######### Perplexity Service ###########
-########################################
+### Perplexity Service ###
 
 # PERPLEXITY_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-########################################
-######### Anthropic Service ############
-########################################
+### Groq Service ####
 
-# ANTHROPIC_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+# GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-########################################
-############ Groq Service ##############
-########################################
+#### 01.AI Service ####
 
-# GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+# ZEROONE_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-########################################
-######### OpenRouter Service ##########
-########################################
+### TogetherAI Service ###
 
-#OPENROUTER_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-#OPENROUTER_CUSTOM_MODELS=model1,model2,model3
+# TOGETHERAI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-########################################
-######### 01.AI Service ##########
-########################################
+### ZhiPu AI ###
 
-#ZEROONE_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+# ZHIPU_API_KEY=xxxxxxxxxxxxxxxxxxx.xxxxxxxxxxxxx
 
-########################################
-######### TogetherAI Service ##########
-########################################
+### Moonshot AI  ####
+
+# MOONSHOT_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
-#TOGETHERAI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
 ########################################
 ############ Market Service ############

diff --git a/Dockerfile b/Dockerfile
@@ -58,13 +58,13 @@ ENV PORT=3210
 
 # General Variables
 ENV ACCESS_CODE ""
-ENV CUSTOM_MODELS ""
 
 ENV API_KEY_SELECT_MODE ""
 
 # OpenAI
 ENV OPENAI_API_KEY ""
 ENV OPENAI_PROXY_URL ""
+ENV OPENAI_MODEL_LIST ""
 
 # Azure OpenAI
 ENV USE_AZURE_OPENAI ""
@@ -82,6 +82,7 @@ ENV MOONSHOT_API_KEY ""
 
 # Ollama
 ENV OLLAMA_PROXY_URL ""
+ENV OLLAMA_MODEL_LIST ""
 
 # Perplexity
 ENV PERPLEXITY_API_KEY ""
@@ -94,7 +95,7 @@ ENV MISTRAL_API_KEY ""
 
 # OpenRouter
 ENV OPENROUTER_API_KEY ""
-ENV OPENROUTER_CUSTOM_MODELS ""
+ENV OPENROUTER_MODEL_LIST ""
 
 # 01.AI
 ENV ZEROONE_API_KEY ""

diff --git a/README.md b/README.md
@@ -128,6 +128,7 @@ We have implemented support for the following model service providers:
 - **Groq**: Accessed Groq's AI models, efficiently processing message sequences and generating responses, capable of multi-turn dialogues and single-interaction tasks. [Learn more](https://groq.com/)
 - **OpenRouter**: Supports routing of models including **Claude 3**, **Gemma**, **Mistral**, **Llama2** and **Cohere**, with intelligent routing optimization to improve usage efficiency, open and flexible. [Learn more](https://openrouter.ai/)
 - **01.AI (Yi Model)**: Integrated the 01.AI models, with series of APIs featuring fast inference speed, which not only shortened the processing time, but also maintained excellent model performance. [Learn more](https://01.ai/)
+- **Together.ai**: Over 100 leading open-source Chat, Language, Image, Code, and Embedding models are available through the Together Inference API. For these models you pay just for what you use. [Learn more](https://www.together.ai/)
 
 At the same time, we are also planning to support more model service providers, such as Replicate and Perplexity, to further enrich our service provider library. If you would like LobeChat to support your favorite service provider, feel free to join our [community discussion](https://github.com/lobehub/lobe-chat/discussions/1284).
 
@@ -459,7 +460,7 @@ This project provides some additional configuration items set with environment v
 | `OPENAI_API_KEY`     | Yes      | This is the API key you apply on the OpenAI account page                                                                                                                  | `sk-xxxxxx...xxxxxx`                                                                                                 |
 | `OPENAI_PROXY_URL`   | No       | If you manually configure the OpenAI interface proxy, you can use this configuration item to override the default OpenAI API request base URL                             | `https://api.chatanywhere.cn` or `https://aihubmix.com/v1` <br/>The default value is<br/>`https://api.openai.com/v1` |
 | `ACCESS_CODE`        | No       | Add a password to access this service; you can set a long password to avoid leaking. If this value contains a comma, it is a password array.                              | `awCTe)re_r74` or `rtrt_ewee3@09!` or `code1,code2,code3`                                                            |
-| `CUSTOM_MODELS`      | No       | Used to control the model list. Use `+` to add a model, `-` to hide a model, and `model_name=display_name` to customize the display name of a model, separated by commas. | `qwen-7b-chat,+glm-6b,-gpt-3.5-turbo`                                                                                |
+| `OPENAI_MODEL_LIST`      | No       | Used to control the model list. Use `+` to add a model, `-` to hide a model, and `model_name=display_name` to customize the display name of a model, separated by commas. | `qwen-7b-chat,+glm-6b,-gpt-3.5-turbo`                                                                                |
 
 > \[!NOTE]
 >

diff --git a/README.zh-CN.md b/README.zh-CN.md
@@ -124,6 +124,7 @@
 - **Anthropic (Claude)**：接入了 Anthropic 的 **Claude** 系列模型，包括 Claude 3 和 Claude 2，多模态突破，超长上下文，树立行业新基准。[了解更多](https://www.anthropic.com/claude)
 - **ChatGLM**：加入了智谱的 **ChatGLM** 系列模型（GLM-4/GLM-4-vision/GLM-3-turbo），为用户提供了另一种高效的会话模型选择。[了解更多](https://www.zhipuai.cn/)
 - **Moonshot AI (月之暗面)**：集成了 Moonshot 系列模型，这是一家来自中国的创新性 AI 创业公司，旨在提供更深层次的会话理解。[了解更多](https://www.moonshot.cn/)
+- **Together.ai**：集成部署了数百种开源模型和向量模型，无需本地部署即可随时访问这些模型。[了解更多](https://www.together.ai/)
 - **01.AI (零一万物)**：集成了零一万物模型，系列 API 具备较快的推理速度，这不仅缩短了处理时间，同时也保持了出色的模型效果。[了解更多](https://www.lingyiwanwu.com/)
 - **Groq**：接入了 Groq 的 AI 模型，高效处理消息序列，生成回应，胜任多轮对话及单次交互任务。[了解更多](https://groq.com/)
 - **OpenRouter**：其支持包括 **Claude 3**，**Gemma**，**Mistral**，**Llama2**和**Cohere**等模型路由，支持智能路由优化，提升使用效率，开放且灵活。[了解更多](https://openrouter.ai/)
@@ -440,7 +441,7 @@ $ docker run -d -p 3210:3210 \
 | `OPENAI_API_KEY`   | 必选 | 这是你在 OpenAI 账户页面申请的 API 密钥                                                                                       | `sk-xxxxxx...xxxxxx`                                                                                   |
 | `OPENAI_PROXY_URL` | 可选 | 如果你手动配置了 OpenAI 接口代理，可以使用此配置项来覆盖默认的 OpenAI API 请求基础 URL                                        | `https://api.chatanywhere.cn` 或 `https://aihubmix.com/v1`<br/>默认值:<br/>`https://api.openai.com/v1` |
 | `ACCESS_CODE`      | 可选 | 添加访问此服务的密码，你可以设置一个长密码以防被爆破，该值用逗号分隔时为密码数组                                              | `awCTe)re_r74` or `rtrt_ewee3@09!` or `code1,code2,code3`                                              |
-| `CUSTOM_MODELS`    | 可选 | 用来控制模型列表，使用 `+` 增加一个模型，使用 `-` 来隐藏一个模型，使用 `模型名=展示名` 来自定义模型的展示名，用英文逗号隔开。 | `qwen-7b-chat,+glm-6b,-gpt-3.5-turbo`                                                                  |
+| `OPENAI_MODEL_LIST`    | 可选 | 用来控制模型列表，使用 `+` 增加一个模型，使用 `-` 来隐藏一个模型，使用 `模型名=展示名` 来自定义模型的展示名，用英文逗号隔开。 | `qwen-7b-chat,+glm-6b,-gpt-3.5-turbo`                                                                  |
 
 > \[!NOTE]
 >

diff --git a/docs/self-hosting/environment-variables/model-provider.mdx b/docs/self-hosting/environment-variables/model-provider.mdx
@@ -64,7 +64,7 @@ Related discussions:
 - [Reasons for errors when using third-party interfaces](https://github.com/lobehub/lobe-chat/discussions/734)
 - [No response in chat after filling in the proxy server address](https://github.com/lobehub/lobe-chat/discussions/1065)
 
-### `CUSTOM_MODELS`
+### `OPENAI_MODEL_LIST`
 
 - Type: Optional
 - Description: Used to control the model list, use `+` to add a model, use `-` to hide a model, use `model_name=display_name` to customize the display name of a model, separated by commas.
@@ -170,6 +170,13 @@ When using the `turn` mode, the API Keys will be retrieved in a round-robin mann
 - Default: -
 - Example: `http://127.0.0.1:11434/v1`
 
+### `OLLAMA_MODEL_LIST`
+
+- Type: Optional
+- Description: Used to customize your own language models
+- Default: -
+- Example: `llama2:7B`
+
 ## Perplexity AI
 
 ### `PERPLEXITY_API_KEY`
@@ -215,7 +222,7 @@ When using the `turn` mode, the API Keys will be retrieved in a round-robin mann
 - Default: -
 - Example: `sk-or-v1-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=`
 
-### `OPENROUTER_CUSTOM_MODELS`
+### `OPENROUTER_MODEL_LIST`
 
 - Type: Optional
 - Description: Used to control the model list, use `+` to add a model, use `-` to hide a model, use `model_name=display_name` to customize the display name of a model, separated by commas.

diff --git a/docs/self-hosting/environment-variables/model-provider.zh-CN.mdx b/docs/self-hosting/environment-variables/model-provider.zh-CN.mdx
@@ -46,7 +46,7 @@ LobeChat 在部署时提供了丰富的模型服务商相关的环境变量，
 - [使用第三方接口报错的原因](https://github.com/lobehub/lobe-chat/discussions/734)
 - [代理服务器地址填了聊天没任何反应](https://github.com/lobehub/lobe-chat/discussions/1065)
 
-### `CUSTOM_MODELS`
+### `OPENAI_MODEL_LIST`
 
 - 类型：可选
 - 描述：用来控制模型列表，使用 `+` 增加一个模型，使用 `-` 来隐藏一个模型，使用 `模型名=展示名` 来自定义模型的展示名，用英文逗号隔开。
@@ -152,6 +152,13 @@ LobeChat 在部署时提供了丰富的模型服务商相关的环境变量，
 - 默认值：-
 - 示例：`http://127.0.0.1:11434/v1`
 
+### `OLLAMA_MODEL_LIST`
+
+- 类型：可选
+- 描述：用于指定自定义 Ollama 语言模型
+- 默认值：-
+- 示例：`qwen:32B`
+
 ## Perplexity AI
 
 ### `PERPLEXITY_API_KEY`
@@ -197,7 +204,7 @@ LobeChat 在部署时提供了丰富的模型服务商相关的环境变量，
 - 默认值：-
 - 示例：`sk-or-v1-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=`
 
-### `OPENROUTER_CUSTOM_MODELS`
+### `OPENROUTER_MODEL_LIST`
 
 - 类型：可选
 - 描述：用来控制模型列表，使用 `+` 增加一个模型，使用 `-` 来隐藏一个模型，使用 `模型名=展示名` 来自定义模型的展示名，用英文逗号隔开。

diff --git a/docs/self-hosting/examples/azure-openai.mdx b/docs/self-hosting/examples/azure-openai.mdx
@@ -1,8 +1,6 @@
 ---
 title: Configuring Azure OpenAI for LobeChat
-description: >-
-  Learn how to configure Azure OpenAI for LobeChat, including usage limitations,
-  interface configuration, and deployment settings.
+description: Learn how to configure Azure OpenAI for LobeChat, including interface configuration, and deployment settings.
 tags:
   - Azure OpenAI
   - LobeChat
@@ -13,15 +11,7 @@ tags:
 
 # Integrating with Azure OpenAI
 
-LobeChat supports using [Azure OpenAI][azure-openai-url] as the model service provider for OpenAI. This article will explain how to configure Azure OpenAI.
-
-## Usage Limitations
-
-Due to development costs ([#178][rfc]), the current version of LobeChat does not fully comply with the implementation model of Azure OpenAI. Instead, it adopts a solution based on `openai` to be compatible with Azure OpenAI. As a result, the following limitations exist:
-
-- Only one of OpenAI and Azure OpenAI can be selected. Once you enable Azure OpenAI, you will not be able to use OpenAI as the model service provider.
-- LobeChat requires the deployment name to be the same as the model name in order to function properly. For example, the deployment name for the `gpt-35-turbo` model must be `gpt-35-turbo`. Otherwise, LobeChat will not be able to match the corresponding model correctly. <Image alt="Usage Limitations" src="https://github-production-user-asset-6210df.s3.amazonaws.com/28616219/267082091-d89d53d3-1c8c-40ca-ba15-0a9af2a79264.png" />
-- Due to the complexity of integrating with Azure OpenAI's SDK, it is currently not possible to query the list of configured models.
+LobeChat supports using [Azure OpenAI](https://learn.microsoft.com/zh-cn/azure/ai-services/openai/concepts/models) as the model service provider for OpenAI. This article will explain how to configure Azure OpenAI.
 
 ## Configuring in the Interface
 
@@ -36,7 +26,7 @@ You can fill in the corresponding configuration items as needed:
 
 - **API Key**: The API key you applied for on the Azure OpenAI account page, which can be found in the "Keys and Endpoints" section.
 - **API Address**: Azure API address, which can be found in the "Keys and Endpoints" section when checking resources in the Azure portal.
-- **Azure API Version**: The API version of Azure, following the format YYYY-MM-DD. Refer to the [latest version][azure-api-verion-url].
+- **Azure API Version**: The API version of Azure, following the format YYYY-MM-DD. Refer to the [latest version](https://learn.microsoft.com/zh-cn/azure/ai-services/openai/reference#chat-completions).
 
 After completing the configuration of the above fields, click "Check". If it prompts "Check passed", it means the configuration was successful.
 
@@ -46,17 +36,7 @@ If you want the deployed version to be pre-configured with Azure OpenAI for end
 
 | Environment Variable | Type | Description | Default Value | Example |
 | --- | --- | --- | --- | --- |
-| `USE_AZURE_OPENAI` | Required | Set this value to `1` to enable Azure OpenAI configuration | - | `1` |
 | `AZURE_API_KEY` | Required | This is the API key you obtained from the Azure OpenAI account page | - | `c55168be3874490ef0565d9779ecd5a6` |
-| `OPENAI_PROXY_URL` | Required | Azure API address, can be found in the "Keys and Endpoints" section when checking resources in the Azure portal | - | `https://docs-test-001.openai.azure.com` |
-| `AZURE_API_VERSION` | Optional | Azure API version, following the format YYYY-MM-DD | 2023-08-01-preview | `2023-05-15`, see [latest version][azure-api-verion-url] |
-| `ACCESS_CODE` | Optional | Add a password to access this service. You can set a long password to prevent brute force attacks. When this value is separated by commas, it becomes an array of passwords | - | `awCT74` or `e3@09!` or `code1,code2,code3` |
-
-<Callout>
-  When you enable `USE_AZURE_OPENAI` on the server, users will be unable to modify and use the
-  OpenAI API key in the frontend configuration.
-</Callout>
-
-[azure-api-verion-url]: https://learn.microsoft.com/zh-cn/azure/ai-services/openai/reference#chat-completions
-[azure-openai-url]: https://learn.microsoft.com/zh-cn/azure/ai-services/openai/concepts/models
-[rfc]: https://github.com/lobehub/lobe-chat/discussions/178
+| `AZURE_ENDPOINT` | Required | Azure API address, can be found in the "Keys and Endpoints" section when checking resources in the Azure portal | - | `https://docs-test-001.openai.azure.com` |
+| `AZURE_API_VERSION` | Optional | Azure API version, following the format YYYY-MM-DD | 2023-08-01-preview | `-`, see [latest version](https://learn.microsoft.com/zh-cn/azure/ai-services/openai/reference#chat-completions) |
+| `ACCESS_CODE` | Optional | Add a password to access LobeChat. You can set a long password to prevent brute force attacks. When this value is separated by commas, it becomes an array of passwords | - | `awCT74` or `e3@09!` or `code1,code2,code3` |