Repository navigation
Supporting out-of-band interactions with script tools #2
Description
Activity
Part of the elicitation API is a feedback loop (progress indication), so that the agent can be informed of user progress that is occurring in another venue. It might sense to expose elicitation concepts in the script tools API so that the agent can be made aware that a user has been requested to take action specifically, and to be informed of progress. I agree that the interaction occurring in the context of the page makes this implicitly knowable, but it might be useful signaling for agents to know this explicitly? It's also an indication to the agent that the site expects the user to be doing something, rather than the agent on their behalf. I think this is roughly aligned with your comment, but the progress indicator mechanism is a slight expansion of the concept.
Reacted by nocluemicheal
MCP is spec-ing a capability for a service to request the MCP client to delegate a request to the user for confirmation here. The goal is to ensure critical decisions, like making payments, are done by the user instead of a tool execution decided by the Agent.
Script tools supports this by design. Since the tool is executed in a web page on the user's browser, the site can elicit user input for executing any tool without any mediation required by the Agent. For example, the site could expose a
makePaymenttool which when invoked brings up the payment flow on the site that requires user interaction.The following cases are still possible:
The user assigns an Agent a task and then switches to a different tab. When executing a tool which requires user interaction, the browser will need to bring this tab into the foreground.
Script tools will likely be used in conjunction with UI actuation. An Agent could incorrectly assume that a tool blocked on user input has hit an error/timed out and fallback to UI actuation.
It will likely be beneficial to include an annotation when a tool needs userInput. So the Agent can bring the tab back to the user's attention. This has some potential for abuse, the site could use the capability to simply get back user attention. The browser can offer UX for the user to indicate if they want the Agent to use to UI actuation. This is similar to user intervention where browser can suppress popups for sites.