Also stuff like "use mmap" or thread count
Originally posted by @binarynoise in #279
There's many ways we can go about this:
- Apply model settings globally. This will mean that all of the models from any server will default to the user-defined parameters. This is how OpenWebUI handles it AFAIK.
- Apply the settings per server/connection. Means that the models from that server will default to the user-defined parameters.
- Apply the settings per model. User-defined settings will only be applied when a particular model is used, all others will use the provider's (Ollama/OpenAI) defaults.
Without having done supe deep research, option 1 is probably the easiest implementation of such a feature.
@binarynoise do you have any insights on how this would play out in your workflow?
There's many ways we can go about this:
Without having done supe deep research, option 1 is probably the easiest implementation of such a feature.
@binarynoise do you have any insights on how this would play out in your workflow?