feat: add per-toolset model routing via model field on toolsets#2015
Open
dgageot wants to merge 1 commit intodocker:mainfrom
Open
feat: add per-toolset model routing via model field on toolsets#2015dgageot wants to merge 1 commit intodocker:mainfrom
dgageot wants to merge 1 commit intodocker:mainfrom
Conversation
Add a 'model' field to toolset configuration that automatically routes the LLM turn processing tool results through a specified model. This enables platform-level cost optimization: cheaper/faster models handle simple tool results (file reads, knowledge-base lookups) while the agent's primary model handles complex reasoning. Config & schema: - Add Model field to Toolset struct in pkg/config/latest/types.go - Add model property to Toolset in agent-schema.json Tool plumbing: - Add ModelOverride field to Tool struct (pkg/tools/tools.go) - Add WithModelOverride wrapper in pkg/teamloader/model_override.go - Wire wrapper into getToolsForAgent in pkg/teamloader/teamloader.go Runtime: - Extract reusable resolveModelRef from SetAgentModel in model_switcher.go with proper alloy config rejection - Add resolveToolCallModelOverride helper (pkg/runtime/tool_model_override.go) - In RunStream loop: after tool calls, record override; on next iteration, temporarily switch model for one turn then reset - Clear override on agent switch to prevent cross-agent leakage Tests: - Unit tests for WithModelOverride wrapper - Unit tests for resolveToolCallModelOverride - Unit tests for resolveModelRef (alloy rejection, nil config, invalid format) - Example config: per_tool_model_routing.yaml Assisted-By: docker-agent
There was a problem hiding this comment.
Review Summary
Assessment: 🟢 APPROVE
This PR adds per-toolset model routing to enable cost optimization by routing simple tool results through cheaper models. The implementation is clean and well-structured:
✅ Strengths:
- Proper separation of concerns: config schema → tool plumbing → runtime integration
- Correct control flow with early returns preventing fall-through bugs
- Comprehensive test coverage for new functions (resolveModelRef, WithModelOverride, resolveToolCallModelOverride)
- Good validation: resolveModelRef properly rejects alloy configs, isInlineAlloySpec validates format
- Clear override lifecycle management (set on tool calls, clear on agent switch)
- Example config demonstrating the feature
🔍 Code Quality:
- The refactoring that extracted resolveModelRef maintains the original behavior correctly
- Alloy spec handling uses mutually exclusive code paths with proper returns
- Model override wrapper pattern is clean and composable
- Error messages are descriptive
No bugs found in the changed code. The feature is ready to merge.
Review completed by cagent
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a 'model' field to toolset configuration that automatically routes the LLM turn processing tool results through a specified model. This enables platform-level cost optimization: cheaper/faster models handle simple tool results (file reads, knowledge-base lookups) while the agent's primary model handles complex reasoning.
Config & schema:
Tool plumbing:
Runtime:
Tests:
Assisted-By: docker-agent