Better support for human-in-the-loop

[ianmacartney]

Currently, to do human-in-the-loop workflows, you can use the AI SDK pattern where you pass in a tool without an execute function. When the tool call happens, it won't continue and the resulting message will be the tool call.
You then save that somewhere that the client has a query on, and when the client submits some data, you can re-execute the thread with the tool response to continue.

If you're doing this as part of a workflow using the Workflow component, however, it's not possible to suspend execution to wait. To have that work, we'll need Pause/Resume

[ianmacartney]

I'm going to add examples soon on how to do this, but punt on doing pause/resume until there's pent-up demand, as things in milestone 2 seem more important

[tonyabracadabra]

Is it supported now?

[ianmacartney]

there's examples of both using a human as a tool call or via other messages here:
https://github.com/get-convex/agent/blob/main/example/convex/chat/human.ts#L1
docs:
https://docs.convex.dev/agents/human-agents

but workflow doesn't have pause/resume yet

[tonyabracadabra]

there's examples of both using a human as a tool call or via other messages here:
https://github.com/get-convex/agent/blob/main/example/convex/chat/human.ts#L1
docs:
https://docs.convex.dev/agents/human-agents

but workflow doesn't have pause/resume yet

Cool! Curious how pause/resume improves the “human as tool” workflow. How is it currently implemented? Does the workflow continue to incur usage charges while waiting for the human to respond?

[ianmacartney]

No workflow would not be running until resumed.
The benefit to "human as tool" is that currently if you have a workflow, you have to end the workflow when you get to the human part, then start a new one. Not the biggest deal, but it'd be nice to have a workflow that stops midway - for a human or other async task

[tonyabracadabra]

No workflow would not be running until resumed.
The benefit to "human as tool" is that currently if you have a workflow, you have to end the workflow when you get to the human part, then start a new one. Not the biggest deal, but it'd be nice to have a workflow that stops midway - for a human or other async task

Got it! Just curious since there’s no usage charge while waiting either way, is the real benefit mainly about not having to spin up a new workflow from the DX perspective? Or is there something else I’m missing?

[ianmacartney]

Yeah just the DX of having one logical function that manages the flow, and not having to have the branching logic of various places it might need to return early for feedback / etc. And not needing to pass out all context, then read it back in

[eliotgevers]

I need exactly this. I am currently working on a "backend agent" for my database. I might ask it: "make a dataset of all astronauts to date". It then works in the background and I can go check on it's progress anytime. At the end I accept/deny the dataset generated.

Sometimes I need to pause to the workflow to give feedback to the agent to help it steer. Or the agent might pause the workflow to notify me that it needs my feedback before it can continue.

Having the ability to pause seems like an important feature to me. Any idea if you would be working on this in the nearby future? Thanks!

[tiagoefreitas]

Hi @ianmacartney Lars is prototyping tool execution approvals for ai sdk, may be relevant here and your input now may influence it positively before it ships.

https://x.com/lgrammel/status/1971219931971846220?s=46&t=PW8PiFwluc0tdmv2tOMdEg

[tiagoefreitas]

Having pause in the workflow would be really useful. I currently don't use the workflow component yet and just restart the my custom "workflow" action when I get an approval, but that incresases complexity