Repository navigation
📬 feat: Per-Call Tool Completion Emission via onResult Channel - #239
Merged
Merged
Conversation
Tool batches execute in parallel on the host, but the ON_TOOL_EXECUTE contract only had a single resolve(results[]) channel — so a fast tool's completion event waited for the slowest call in the batch. Add an optional onResult callback to ToolExecuteBatchRequest: the host may report each result as it settles (before the final resolve), and ToolNode emits that call's completed run step immediately. Purely an emission fast-path — resolve remains the authoritative batch outcome, graph state still transitions once per batch. - Offered only in no-hooks/no-HITL configurations: PostToolUse hooks can rewrite output and HITL can deny a call after execution, so those keep batch-time emission - Ids are claimed synchronously to dedupe duplicate reports; unknown ids are ignored; failed dispatches release the claim so the batch path re-emits - Output formatting mirrors the batch loop exactly (truncation and error formatting), and dispatchStepCompleted now reports whether the event was delivered
7 tasks done
Collaborator
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
I decoupled tool-completion UI latency from the slowest call in a batch. The host already executes tool batches in parallel (LibreChat's
createToolExecuteHandlerrunsPromise.allover the calls), but theON_TOOL_EXECUTEcontract only exposed a singleresolve(results[])channel — so a 1-second tool's completion event waited for the 30-second tool next to it. This adds an optional per-call channel: the host may report each result throughonResultthe moment it settles, and ToolNode emits that call's completed run step immediately. It is purely an emission fast-path —resolveremains the authoritative batch outcome, the graph state still transitions once per batch (the next model turn inherently needs every ToolMessage), and handlers that never callonResultbehave exactly as today. This composes with the eager-execution seals from #237/#238 rather than overlapping them: sealing moves when execution starts;onResultmoves when results appear, including on the fully lazy path.onResult?: (result: ToolExecuteResult) => voidtoToolExecuteBatchRequest, documented as optional and non-authoritative.PostToolUsehooks can rewrite output (updatedOutput) and HITL can deny a call after execution, so those configurations keep batch-time emission; this mirrors the existing batch-sensitivity gate for eager execution.onResultcalls; ignored unknown ids; released the claim when a dispatch fails or is not delivered so the batch path re-emits.dispatchStepCompletedto report whether the event was delivered.data.onResult?.(result)inside the existingPromise.allmapper; safe to land before this releases since the callback is optional.Change Type
Testing
src/tools/__tests__/ToolNode.onResultCompletion.test.ts(6 tests): per-call emission lands before the batch resolves with no double emission; duplicate/unknown reports ignored;onResultwithheld when hooks or HITL are configured; undelivered early dispatch falls back to batch emission; error results use the standard error formatting.npx jest src/tools src/__tests__ src/stream.test.ts— 956 tests pass;npx tsc --noEmit, eslint, prettier, sort-imports clean.Test Configuration:
Checklist