feature request: tool calling - 'toolChoice'

[jakobhoeg]

Hey! I love what you're building here and I am currently working on a model provider for Vercel AI SDK here.

The AI SDK lets you set a 'toolChoice' which tells the model if it should use tools or not. By default it is set to 'auto', meaning the model will use tools when appropiate - which by my understanding will be default for the Prompt API too.
But it can also be set to 'none' or 'required'. See this

I didn't see this mentioned in the readme here, so just wanted to ask if it is going to be included in the Prompt API?

[domenic]

Thank you for the kind words! Yeah, this seems like a reasonable feature request to me, if the models support it.

Do you have any cases you know of where this has been crucial for developers in the wild? That will help us prioritize it.

Also, @clarkduvall and @sushraja-msft, can you confirm that this is implementable with the models that currently back your prompt API implementations?

[jakobhoeg]

Thank you for the kind words! Yeah, this seems like a reasonable feature request to me, if the models support it.

Do you have any cases you know of where this has been crucial for developers in the wild? That will help us prioritize it.

Also, @clarkduvall and @sushraja-msft, can you confirm that this is implementable with the models that currently back your prompt API implementations?

While I don't have a case in mind where this has been crucial (I'm sure others might be able to provide cases), I think it does give developers better control over the model.
Most API's that are OpenAI compatibile also allow usage of this as well, so might make sense to atleast have it in mind for the Prompt API too (if applicable)

[sushraja-msft]

The underlying model Phi4 does not explicitly support gradations in tool calling eagerness. However, I think this can be achieved using system prompt.

As part of implementing tool calling browser implementations will have to append a system message "You are a helpful assistant with some tools.". The toolchoice parameter can be used to change that message to "You are a helpful assistant with some tools. Use them in every turn". We would need more analysis to see how effective such system message changes are in enforcing behavior.

One challenge we will run into is that if the declared tools are such that they require parameters, forcing the model to call them will result in the model asking the user questions to fill out the parameters rather than making the tool call.