Skip to content

Support custom request params#116

Open
sainnhe wants to merge 1 commit intoggml-org:masterfrom
sainnhe:feature/custom-params
Open

Support custom request params#116
sainnhe wants to merge 1 commit intoggml-org:masterfrom
sainnhe:feature/custom-params

Conversation

@sainnhe
Copy link
Copy Markdown

@sainnhe sainnhe commented Mar 1, 2026

Issue Description

I'm using this plugin along with qwen3.5, and I want to serve 1 model to handle both fim and chat tasks.

For fim tasks, I want to set repeat-penalty to 1.0, and for chat tasks I want to set it to 1.05. Since the built in web page of llama-server doesn't seem to support custom params, so I need to pass --repeat-penalty to 1.05 to llama-server. But if I do so, I cannot use repeat-penalty=1.0 in llama.vim since this plugin doesn't provide any method to allow users set custom request params.

The Solution

This PR adds 2 config options: fim_params and inst_params. Users can set custom request params via these options. I use extend(..., 'force') so user's config is always prior to the default params.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant