Skip to content

Fix MCP test independence #5046

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fix MCP test independence #5046

wants to merge 2 commits into from

Conversation

daniel-lxs
Copy link
Collaborator

@daniel-lxs daniel-lxs commented Jun 23, 2025

Ensures each MCP test sets up its own configuration independently by adding alwaysAllow permissions setup to all tests. This prevents test failures when tests run in different orders.


Important

Ensures MCP test independence by setting up individual configurations and introduces mcpServersInitialized event for server initialization handling.

  • Behavior:
    • Ensures each MCP test in use-mcp-tool.test.ts sets up its own configuration independently by modifying the mcpConfig to include alwaysAllow permissions.
    • Adds logic to modify and save the MCP config file to trigger server detection in use-mcp-tool.test.ts.
    • Introduces a 1-second delay after modifying the config to allow server initialization.
  • Events:
    • Adds mcpServersInitialized event to RooCodeAPIEvents in api.ts and ipc.ts.
    • Emits mcpServersInitialized in McpHub after server connections are updated.
  • Misc:
    • Extends ClineProviderEvents in ClineProvider.ts to include mcpServersInitialized.
    • Updates API in api.ts to listen for mcpServersInitialized and emit it.

This description was created by Ellipsis for e52478f. You can customize this summary. It will automatically update as commits are pushed.

@daniel-lxs daniel-lxs requested review from mrubens, cte and jr as code owners June 23, 2025 15:30
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Jun 23, 2025
@daniel-lxs
Copy link
Collaborator Author

This seems to help @cte @mrubens

@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Review] in Roo Code Roadmap Jun 23, 2025
@@ -841,7 +940,40 @@ suite("Roo Code use_mcp_tool Tool", function () {
}
}
api.on("taskCompleted", taskCompletedHandler)
await sleep(2000) // Wait for Roo Code to fully initialize
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we hook into our event emitter to determine if we're initialized instead of sleeping?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like taskStarted will not wait for MCP servers to be initialized causing the model to not see them.

Do you know of any event I can use to wait for the MCP servers to initialize?

interface RooCodeAPIEvents {
    message: [data: {
        taskId: string;
        action: "created" | "updated";
        message: ClineMessage;
    }];
    taskCreated: [taskId: string];
    taskStarted: [taskId: string];
    taskModeSwitched: [taskId: string, mode: string];
    taskPaused: [taskId: string];
    taskUnpaused: [taskId: string];
    taskAskResponded: [taskId: string];
    taskAborted: [taskId: string];
    taskSpawned: [parentTaskId: string, childTaskId: string];
    taskCompleted: [taskId: string, tokenUsage: TokenUsage, toolUsage: ToolUsage, isSubtask: IsSubtask];
    taskTokenUsageUpdated: [taskId: string, tokenUsage: TokenUsage];
    taskToolFailed: [taskId: string, toolName: ToolName, error: string];
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add one?

@daniel-lxs daniel-lxs moved this from PR [Needs Review] to PR [Changes Requested] in Roo Code Roadmap Jun 23, 2025
Copy link

delve-auditor bot commented Jun 25, 2025

We have finished reviewing your PR. We have found no vulnerabilities.

Reply to this PR with @delve-auditor followed by a description of what change you want and we'll auto-submit a change to this PR to implement it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working PR - Changes Requested size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: PR [Changes Requested]
Development

Successfully merging this pull request may close these issues.

3 participants