Add built-in support for tool control parameters #347
Conversation
- Implement provider-specific tool choice handling - Add InvalidToolChoiceError for validation - Enhance tool execution flow to prevent infinite loops with non-auto choices
docs/_core_features/tools.md
Outdated
| # Choice options | ||
| chat.with_tool(Weather, choice: :auto) # Model decides whether to call any provided tools or not (default) | ||
| chat.with_tool(Weather, choice: :any) # Model must use one of the provided tools | ||
| chat.with_tool(Weather, choice: :none) # No tools | ||
| chat.with_tool(Weather, choice: :weather) # Force specific tool | ||
|
|
||
| # Parallel tool calls | ||
| chat.with_tools(Weather, Calculator, parallel: true) # Model can output multiple tool calls at once (default) | ||
| chat.with_tools(Weather, Calculator, parallel: false) # At most one tool call | ||
| ``` |
There was a problem hiding this comment.
Both examples of with_tool and with_tools should contain both choice and parallel parameters otherwise people get the false sense that one parameter is for one call only.
lib/ruby_llm/chat.rb
Outdated
| halt_result || complete(&) | ||
| return halt_result if halt_result | ||
|
|
||
| should_continue_after_tools? ? complete(&) : response |
There was a problem hiding this comment.
The solution to this is not to halt the conversation, but to reset tool_choice to auto after tools have been called.
Halting the conversation is not normal behavior and it's for very specific use cases. More info here: https://community.openai.com/t/infinite-loop-with-tool-choice-required-or-type-function/755129
lib/ruby_llm/chat.rb
Outdated
| @@ -205,6 +215,26 @@ def execute_tool(tool_call) | |||
| tool.call(args) | |||
| end | |||
|
|
|||
| def update_tool_options(choice:, parallel:) | |||
| unless choice.nil? | |||
| valid_tool_choices = %i[auto none any] + tools.keys | |||
There was a problem hiding this comment.
as mentioned in the issue, I want the API to look like auto, none or required as required captures better what's going to happen.
lib/ruby_llm/chat.rb
Outdated
| @@ -5,7 +5,7 @@ module RubyLLM | |||
| class Chat | |||
| include Enumerable | |||
|
|
|||
| attr_reader :model, :messages, :tools, :params, :headers, :schema | |||
| attr_reader :model, :messages, :tools, :tool_choice, :parallel_tool_calls, :params, :headers, :schema | |||
There was a problem hiding this comment.
Better to compress these two into one tool_prefs = {choice: ..., parallel: ...}
There was a problem hiding this comment.
Should choice and parallel default to nil (like temperature) to use provider defaults, or should we set them explicitly to :auto and true?
| Control when and how tools are called using `choice` and `parallel` options. | ||
|
|
||
| **Parameter Values:** | ||
| - **`choice`**: Controls tool choice behavior | ||
| - `:auto` Model decides whether to use any tools | ||
| - `:required` - Model must use one of the provided tools | ||
| - `:none` - Disable all tools | ||
| - `"tool_name"` - Force a specific tool (e.g., `:weather` for `Weather` tool) | ||
| - **`parallel`**: Controls parallel tool calls | ||
| - `true` Allow multiple tool calls simultaneously | ||
| - `false` - One at a time | ||
|
|
||
| If not provided, RubyLLM will use the provider's default behavior for tool choice and parallel tool calls. | ||
|
|
||
| **Examples:** | ||
|
|
||
| ```ruby | ||
| chat = RubyLLM.chat(model: 'gpt-4o') | ||
|
|
||
| # Basic usage with defaults | ||
| chat.with_tools(Weather, Calculator) # uses provider defaults | ||
|
|
||
| # Force tool usage, one at a time | ||
| chat.with_tools(Weather, Calculator, choice: :required, parallel: false) | ||
|
|
||
| # Force specific tool | ||
| chat.with_tool(Weather, choice: :weather, parallel: true) |
There was a problem hiding this comment.
I'd prefer that to be code first, then parameter values. Also when specifying a tool in choice, it should accept the tool class too.
| @tool_prefs = { choice: nil, parallel: nil } | ||
| @messages = [] | ||
| @tools = {} |
There was a problem hiding this comment.
small thing, but make it after @tools
| @@ -130,6 +135,7 @@ def complete(&) # rubocop:disable Metrics/PerceivedComplexity | |||
| params: @params, | |||
| headers: @headers, | |||
| schema: @schema, | |||
| tool_prefs: @tool_prefs, | |||
What this does
Adds tool choice and parallel tool calls options to the
with_toolandwith_toolsmethod.Users can now specify:
choice: control how the model should use tools (auto,none,required, or specific tool name)parallel: boolean parameter to allow/disallow parallel tool callsAlso updates the
handle_tool_callsmethod. With :required or specific tool choices, the tool_choice is automatically reset tonilafter tool execution to prevent infinite loops.Type of change
Scope check
Quality check
overcommit --installand all hooks passmodels.json,aliases.json)API changes
Related issues
Fixes #343