Skip to content

Add model_picker toolset for dynamic model switching#1937

Merged
dgageot merged 1 commit intodocker:mainfrom
dgageot:selector
Mar 5, 2026
Merged

Add model_picker toolset for dynamic model switching#1937
dgageot merged 1 commit intodocker:mainfrom
dgageot:selector

Conversation

@dgageot
Copy link
Member

@dgageot dgageot commented Mar 5, 2026

Introduce a new 'model_picker' toolset that lets agents switch between LLM models mid-conversation. This enables agents to pick the best model for each sub-task (e.g. a fast model for simple questions, a powerful one for complex reasoning) and revert when done.

Changes:

  • Add ModelPickerTool with change_model and revert_model tools
  • Register model_picker in the toolset registry
  • Wire runtime handlers (handleChangeModel, handleRevertModel) that validate the requested model against an allowed list, update the agent's model, and emit AgentInfo events so the UI reflects changes
  • Simplify registerDefaultTools to a plain handler map instead of pre-resolving tool definitions
  • Add config validation: 'models' field is required and exclusive to model_picker toolsets
  • Update agent-schema.json with model_picker type and models field
  • Add example configuration (examples/model_picker.yaml)
  • Add comprehensive unit tests for ModelPickerTool

Introduce a new 'model_picker' toolset that lets agents switch between
LLM models mid-conversation. This enables agents to pick the best model
for each sub-task (e.g. a fast model for simple questions, a powerful
one for complex reasoning) and revert when done.

Changes:
- Add ModelPickerTool with change_model and revert_model tools
- Register model_picker in the toolset registry
- Wire runtime handlers (handleChangeModel, handleRevertModel) that
  validate the requested model against an allowed list, update the
  agent's model, and emit AgentInfo events so the UI reflects changes
- Simplify registerDefaultTools to a plain handler map instead of
  pre-resolving tool definitions
- Add config validation: 'models' field is required and exclusive to
  model_picker toolsets
- Update agent-schema.json with model_picker type and models field
- Add example configuration (examples/model_picker.yaml)
- Add comprehensive unit tests for ModelPickerTool

Signed-off-by: David Gageot <david.gageot@docker.com>
@dgageot dgageot requested a review from a team as a code owner March 5, 2026 09:20
Copy link

@docker-agent docker-agent bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Assessment: 🟡 NEEDS ATTENTION

The model_picker implementation is well-structured overall, but there's a validation gap that could allow invalid model configurations to pass through to runtime.

Found 1 issue requiring attention:

  • 1 MEDIUM severity issue in config validation

Key Finding

The validation logic checks that the models list is non-empty for model_picker toolsets, but it doesn't validate the contents of that list. This means configurations with empty strings (e.g., models: [""]) would pass validation but cause runtime errors when the agent attempts to switch models.

Recommendation

Add validation to iterate over t.Models and ensure each entry is a non-empty string with valid model identifier format.

return errors.New("openapi toolset requires a url to be set")
}
case "model_picker":
if len(t.Models) == 0 {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MEDIUM: Missing validation for individual model strings

The validation checks that len(t.Models) > 0, but it doesn't validate whether each string in the Models slice is non-empty or valid. A configuration like this would pass validation:

model_picker:
  models:
    - ""
    - "gpt-4"

But would cause runtime errors when the agent tries to switch to the empty model string. Consider adding:

for i, model := range t.Models {
    if strings.TrimSpace(model) == "" {
        return fmt.Errorf("toolset %q: models[%d] cannot be empty", t.Name, i)
    }
}

@dgageot dgageot merged commit 466d846 into docker:main Mar 5, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants