Make adding other OpenAI-Compatible providers possible

[lex-ibm]

No description provided.

[En3Tho]

@lex-ibm I don't see anything mentioning api key for custom provider. Is it implied somehow by the existing codebase?

[lex-ibm]

@En3Tho the API key part of it is already managed by the parent class BaseOpenAICompatibleBYOKRegistry, in fact, I just realized that maybe I could just use that one instead

[lex-ibm]

My bad... BaseOpenAICompatibleBYOKRegistry is an abstract class

[En3Tho]

I think what you're doing already is correct, yeah.

[relic-yuexi]

I tested this modification, and it works well, but there are a few details.

    "github.copilot.chat.openAICompatibleProviders": [
        {
            "name": "douchat",
            "url": "https://xxx/v1",
        }
    ],

After v1, there shouldn't be an additional /, or it will cause an error.

Also, I wasn't sure about the model requirements for the agent, so I wasn't able to add it successfully. However, both the edit and ask modes work fine.

I got it. It need toolcalling ablility.

Thank you very much for your work — it's amazing!

[relic-yuexi]

@lex-ibm

Hello lex-ibm, i have make a pr to your project. Now we can use UI to control more provider easily.

You can download it in: https://drive.google.com/file/d/1yFuBnmqmVXvDhv3cPamJy5g8ynR2tjrH/view?usp=sharing

[isidorn]

@lex-ibm thanks a lot for this PR 🙏 @lramos15 is the right person to review this one, but he is out on vacation this week. So some initial comments from me:

1. Did you get a chance to test this out? You should be able to build the Copilot chat extension https://github.com/microsoft/vscode-copilot-chat/blob/main/CONTRIBUTING.md and then be able to confirm it works with some other providers (for example LM Studio)
2. I do not think this should be a setting, but instead the Manage Models dropdown should have a "Custom Endpoint (OpenAI compatible)" entry. Then a user can input the values you introduced in the setting. You remember that in local storage. This way it will be discoverable to the users (in the UI flow), and will not pollute user's settings

[relic-yuexi]

@lex-ibm thanks a lot for this PR 🙏 @lramos15 is the right person to review this one, but he is out on vacation this week. So some initial comments from me:

1. Did you get a chance to test this out? You should be able to build the Copilot chat extension https://github.com/microsoft/vscode-copilot-chat/blob/main/CONTRIBUTING.md and then be able to confirm it works with some other providers (for example LM Studio)
2. I do not think this should be a setting, but instead the Manage Models dropdown should have a "Custom Endpoint (OpenAI compatible)" entry. Then a user can input the values you introduced in the setting. You remember that in local storage. This way it will be discoverable to the users (in the UI flow), and will not pollute user's settings

@isidorn Hi isidorn, here is my pr lex-ibm#1. And i have do what you said. Like not setting. And in the release: https://github.com/relic-yuexi/vscode-copilot-chat/releases/tag/0.1 you can install it to test. I have test it can work for me.

[isidorn]

@relic-yuexi thank you for offering. However let's first give some time for @lex-ibm to respond, as he is the initial author of the PR.

[rasyidrafi]

Nice feature! I didn't know this Copilot Chat extension is open source, which makes this possible. I thought it was closed source.

[zhuangqh]

+1 for this feature.

[granbestiapop]

Avoid layoffs merging amazing PRs. Nice feature!

[headkit]

+1 from my side

[lex-ibm]

Sorry, I was on vacations last week. I'll take a look at @relic-yuexi's PR and the feedback provided.

[zhuangqh]

When I use the vsix compiled from this change and configure the openai compatible server. The model doesn't work.
It gives me this error when i chat with copilot. Do you guys meet this issue?

Sorry, no response was returned.

[relic-yuexi]

When I use the vsix compiled from this change and configure the openai compatible server. The model doesn't work. It gives me this error when i chat with copilot. Do you guys meet this issue?

Sorry, no response was returned.

In my try.

vLLM hermas tool-call-parser can't work.

sgalng qwen2 tool-call-parser is good!

one-api seems can't use tool, you can find in this https://github.com/songquanpeng/one-api/issues?q=is%3Aissue%20tool-call

python -m sglang.launch_server --model-path Qwen/Qwen3-32B --reasoning-parser qwen3  --served-model-name qwen332b  --tp 8 --tool-call-parser qwen25

can use agent, but

vllm serve Qwen/Qwen3-32B \
    --tensor-parallel-size 8 \
    --enable_prefix_caching \

    --served-model-name qwen332b \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --reasoning-parser deepseek_r1

can't.

[zhuangqh]

When I use the vsix compiled from this change and configure the openai compatible server. The model doesn't work. It gives me this error when i chat with copilot. Do you guys meet this issue?

Sorry, no response was returned.

In my try.

vLLM hermas tool-call-parser can't work.

sgalng qwen2 tool-call-parser is good!

one-api seems can't use tool, you can find in this https://github.com/songquanpeng/one-api/issues?q=is%3Aissue%20tool-call

python -m sglang.launch_server --model-path Qwen/Qwen3-32B --reasoning-parser qwen3  --served-model-name qwen332b  --tp 8 --tool-call-parser qwen25

can use agent, but

vllm serve Qwen/Qwen3-32B \
    --tensor-parallel-size 8 \
    --enable_prefix_caching \

    --served-model-name qwen332b \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --reasoning-parser deepseek_r1

can't.

did you find any model served by vllm can be used for this? I tried llama3.1, phi, qwen2.5-coder with vllm, none of them work.

[relic-yuexi]

you should try sglang with qwen25 tool-parser. vllm hermas has some error ---- Replied Message ---- From ***@***.***> Date 07/09/2025 08:02 To ***@***.***> Cc ***@***.***>***@***.***> Subject Re: [microsoft/vscode-copilot-chat] Make adding other OpenAI-Compatible providers possible (PR #38) zhuangqh left a comment (microsoft/vscode-copilot-chat#38) When I use the vsix compiled from this change and configure the openai compatible server. The model doesn't work. It gives me this error when i chat with copilot. Do you guys meet this issue? Sorry, no response was returned. In my try. vLLM hermas tool-call-parser can't work. sgalng qwen2 tool-call-parser is good! one-api seems can't use tool, you can find in this https://github.com/songquanpeng/one-api/issues?q=is%3Aissue%20tool-call python -m sglang.launch_server --model-path Qwen/Qwen3-32B --reasoning-parser qwen3 --served-model-name qwen332b --tp 8 --tool-call-parser qwen25 can use agent, but vllm serve Qwen/Qwen3-32B \ --tensor-parallel-size 8 \ --enable_prefix_caching \ --served-model-name qwen332b \ --enable-auto-tool-choice \ --tool-call-parser hermes \ --reasoning-parser deepseek_r1 can't. did you find any model served by vllm can be used for this? I tried llama3.1, phi, qwen2.5-coder with vllm, none of them work. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[zhuangqh]

@relic-yuexi I figure out the cause why vllm server can't be used as a custom endpoint. I create an issue here.
microsoft/vscode#255934

[isidorn]

Before reviewing and potentially merging this PR we want to first finalize the Language Model Provider API - this should happen in the next couple of weeks microsoft/vscode#250007

Thanks for your patience 🙏