Skip to content

Configure default parameters for models #307

@fmaclen

Description

@fmaclen

Also stuff like "use mmap" or thread count
Originally posted by @binarynoise in #279

There's many ways we can go about this:

  1. Apply model settings globally. This will mean that all of the models from any server will default to the user-defined parameters. This is how OpenWebUI handles it AFAIK.
  2. Apply the settings per server/connection. Means that the models from that server will default to the user-defined parameters.
  3. Apply the settings per model. User-defined settings will only be applied when a particular model is used, all others will use the provider's (Ollama/OpenAI) defaults.

Without having done supe deep research, option 1 is probably the easiest implementation of such a feature.

@binarynoise do you have any insights on how this would play out in your workflow?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions