Repository navigation
Conversation
|
Will review soon if this is aligned with what I had in mind. There are some architectural decisions here that may need to be changed, too |
|
I believe that we should use MCP elicitation feature for this purpose. It's part of the MCP spec and allows more advanced features like user input of information that might even skip the LLM entirely. |
|
@jjhidalgar-celonis I think both have its place. I understand approval under this PR will happen before the tool is called (before generated content is sent to MCP), but the elicitation usually happens as confirmation after call before MCP makes some definitive action (e.g. sending e-mail or writing in DB). There was open PR for elicitation, but it was not merged, which is a pity. |
Our compliance needs would require data to be manually approved before sending to the MCP - for us, elicitation wouldn't be sufficient |
Ports the tool approval feature from aron/tool-approval branch onto the latest codebase. Adds manual user approval flow for tool calls before execution, configurable via librechat.yaml toolApproval config. Key changes: - Add TToolApproval schema to data-provider config (required/excluded patterns) - Add approval.ts utilities (requiresApproval, matchesPattern, getToolServerName) - Add MCPToolCallValidationHandler for flow-based approval via FlowStateManager - Wrap non-MCP tools with approval in ToolService.loadAgentTools - Add MCP tool validation in MCP.js createToolInstance - Handle native Anthropic web search approval in callbacks.js - Disable native web_search when approval required (OpenAI initialize) - Add validation SSE delta handling in useStepHandler - Add approve/reject UI in ToolCall.tsx with confirm/reject API calls - Add validation routes: POST /api/mcp/validation/confirm|reject/:id - Add i18n keys for approval UI - Add toolApproval example config in librechat.example.yaml Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nfig The validation flow in MCP.js was running unconditionally for all MCP tool calls. Now checks requiresApproval() against the toolApproval config using the full tool key (toolName_mcp_serverName) before initiating the approval flow. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Native Anthropic web_search executes server-side before we can intercept it, making pre-execution approval impossible. Removing all native web search approval code (dropParams in OpenAI/Anthropic initialize, handleNativeWebSearchApproval in callbacks, toolApprovalConfig in getDefaultHandlers). Tool approval remains fully functional for MCP tools and LangChain tools. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7554037 to
b7d0866
Compare
Code Review — Validated FindingsI audited the full diff against the current codebase. Here's a refined list of findings with context and suggested fixes. Organized by priority. Must Fix (4 MAJOR)1. Race Condition: SSE Emitted Before Flow State is StoredFiles: Both
If the confirm route is hit between steps 2 and 3, The OAuth flow gets this right — it uses Suggested fix: Call // 1. Store PENDING state first
await flowManager.initFlow(validationId, validationFlowType, flowMetadata);
// 2. Now safe to tell the client
sendEvent(res, { event: GraphEvents.ON_RUN_STEP_DELTA, data: validationData });
// 3. Monitor the existing flow
await flowManager.createFlow(validationId, validationFlowType, flowMetadata, derivedSignal);Apply this to both Practical risk: The race window (~250ms) is shorter than human reaction time, so a user clicking manually is unlikely to trigger it. But it's an architectural defect that automated clients or high-latency keyv backends could hit, and the fix is straightforward. 2. TTL Mismatch: UI Shows 10 Minutes, Backend Times Out at 3 MinutesFiles: Three disconnected values:
The user sees a 10-minute approval window, but the backend flow manager rejects after 3 minutes. An approval submitted at minute 4 fails with no clear explanation.
Suggested fix: Either:
Whichever path you choose, remove the dead 3. Non-MCP Tool Approval Bypassed in Event-Driven Execution PathFiles:
const needsApproval = requiresApproval(tool.name, toolApprovalConfig);
if (res && needsApproval && tool.mcp !== true) {
tool = wrapToolWithApproval({ tool, res, streamId });
}
MCP tools are unaffected since their approval check lives inside Suggested fix: Add the same 4. Zero Tests for Core Approval LogicFiles: No test files were added in this PR. The security-critical gating functions ( What needs tests:
Should Fix (5 MINOR)5.
|
| # | File | Issue | Fix |
|---|---|---|---|
| 10 | approval.ts |
getBaseToolName exported but never imported anywhere |
Remove or mark with a TODO for intended future use |
| 11 | ToolService.js ~line 832 |
const config = parentConfig; useless alias |
Use parentConfig directly |
| 12 | ToolCall.tsx ~line 320 |
validation != null && validation && redundant |
Simplify to validation && |
| 13 | handler.ts line 81 |
validationId is fully predictable (userId:server:tool:timestamp) |
Low risk since routes are JWT+ownership protected, but worth noting |
| 14 | translation.json |
com_ui_confirming key name vs. "Approving..." value |
Rename key to com_ui_approving or change value to "Confirming..." |
Note on a Prior Review Claim (MCP Pattern Matching)
A prior review claimed that individual MCP tool patterns like send_gmail_message_mcp_* silently fail because Constants.mcp_delimiter is :::mcp:::. This is incorrect. The actual value is Constants.mcp_delimiter = '_mcp_' (see packages/data-provider/src/config.ts:1882). Tool keys are constructed as toolName_mcp_serverName, so the documented patterns work correctly via the startsWith prefix check. The :::mcp::: references in approval.ts are backward-compat checks, not the current format. Disregard that claim.
- Fix race condition: call flowManager.initFlow() before SSE emit - Fix TTL mismatch: align expires_at with backend 3-minute timeout - Add approval wrapping to loadToolsForExecution for non-MCP tools - Fix completeValidationFlow: remove failFlow from catch block - Remove dead code: generateState(), FLOW_TTL, getFlowTTL(), getBaseToolName - Remove unused expires_at prop from ToolCall/Part components - Cache toolApprovalConfig lookup per tool instance in MCP.js - Fix malformed JSDoc, config alias, redundant checks, translation key - Add tests: approval.spec.ts, handler.spec.ts, ToolCall validation tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Thanks for the thorough review @danny-avila ! All findings have been addressed in the latest push: Must Fix (4 Major):
Should Fix (5 Minor):
Nits:
|
|
@codex review |
|
@aron-muon fix eslint issues |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9687f249f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…rettier - Add random nonce to validation IDs to prevent same-millisecond collisions - Include userId and tenantId in getAppConfig call for MCP approval checks - Fix prettier formatting in ToolService.js and ToolCall.test.tsx Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All fixed! @danny-avila |
|
I'm marking this PR as a draft and will keep it as a reference for the official implementation. The current architecture doesn't line up with the event demarcation and control model that's about to become native to I understand the governance and compliance motivations behind both this approach, and it will still be addressed. With the upcoming changes, a pre-tool-use approval will touch far fewer files in the core, and the same extension points will cover future features like code execution and agent skills, without needing to patch the codebase each time. Thanks @aron-muon for the thorough iteration here, the review rounds will directly inform the native version. |
…upstream # Conflicts: # api/server/services/ToolService.js # packages/data-provider/src/config.ts
any ideas why elicitation wasn't merged or if there is a track to get that merged? |
|
continuing here: #12938 |
Summary
Adds a manual user approval flow for agent tool calls. When configured, tool execution is paused until the user explicitly approves or rejects the call via the chat UI. This provides a human-in-the-loop safety mechanism for sensitive or high-impact tool operations.
Scope: MCP tools and LangChain tools only. Native provider tools (e.g. Anthropic's built-in
web_search) execute server-side and cannot be intercepted, so they are excluded from this feature.Closes #5580
Configuration
Add
toolApprovalunderendpoints.agentsinlibrechat.yaml:Pattern matching supports:
"conversations_add_message_mcp_slack""mcp:*"(all MCP tools),"create*_mcp_atlassian"(prefix match)"all"(matches everything)MCP tool names follow the format
{toolName}_mcp_{serverName}— e.g.send_gmail_message_mcp_google,createJiraIssue_mcp_atlassian.Architecture
Backend flow:
requiresApproval()checks the tool name against thetoolApprovalconfig using pattern matchingFlowStateManagerwith a uniquevalidationIdvalidation: validationIdis sent to the clientTwo interception points:
createToolInstanceinMCP.jsusing the fulltoolName_mcp_serverNamekeywrapToolWithApproval()inToolService.jsFrontend flow:
useStepHandlerpropagatesvalidationfield from SSE deltas to tool call content partsToolCall.tsxrenders Approve/Reject buttons whenvalidationis presentPOST /api/mcp/validation/confirm|reject/:validationIdFiles Changed
data-provider/src/config.tstoolApprovalSchema,TToolApprovaltypedata-provider/src/types/agents.tsvalidationfield onToolCall,ToolCallDeltadata-provider/src/types/assistants.tsvalidationfield onPartMetadatapackages/api/src/tools/approval.tsrequiresApproval(),matchesPattern(),getToolServerName()packages/api/src/mcp/validation/handler.tsMCPToolCallValidationHandler(flow-based approval)api/server/services/ToolService.jswrapToolWithApproval(), approval check inloadAgentToolsapi/server/services/MCP.jscreateToolInstancefor MCP toolsapi/server/routes/mcp.jsPOST .../confirm/:id,POST .../reject/:id,GET .../status/:idclient/src/hooks/SSE/useStepHandler.tsvalidationfrom SSE deltasclient/src/components/.../Part.tsxvalidationprop toToolCallclient/src/components/.../ToolCall.tsxlibrechat.example.yamltoolApprovalconfigclient/src/locales/en/translation.jsonKnown Limitations
web_search(identified bysrvtoolu_IDs) executes on Anthropic's servers before the response reaches LibreChat. Pre-execution approval is not possible for these tools.Change Type
Testing
toolApprovalconfig tolibrechat.yamlunderendpoints.agentsrequired: true(all tools) and pattern arraysexcludedpatterns to verify read tools skip approvalChecklist