Skip to content

Fix smoke-copilot: call Serena tools directly instead of inferring availability from mcp-inspect #1249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lpcox merged 2 commits into from
Feb 21, 2026
Merged

Fix smoke-copilot: call Serena tools directly instead of inferring availability from mcp-inspect #1249

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
44505ab
Initial plan
Copilot Feb 21, 2026
c9699ba
Fix smoke-copilot: instruct agent to call Serena tools directly witho…
Copilot Feb 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/smoke-copilot.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,9 @@ timeout-minutes: 15
1. **GitHub MCP Testing**: Review the last 2 merged pull requests in ${{ github.repository }}
2. **Safe Inputs GH CLI Testing**: Use the `safeinputs-gh` tool to query 2 pull requests from ${{ github.repository }} (use args: "pr list --repo ${{ github.repository }} --limit 2 --json number,title,author")
3. **Serena MCP Testing**:
- Use the `serena-activate_project` tool to initialize the workspace at `${{ github.workspace }}` and verify it succeeds (do NOT use bash to run go commands - use Serena's MCP tools or the safeinputs-go/safeinputs-make tools from the go-make shared workflow)
- After initialization, use the `serena-find_symbol` tool to search for symbols and verify that at least 3 symbols are found in the results
- Call the `serena-activate_project` tool DIRECTLY with `path: "${{ github.workspace }}"` — do NOT use `mcp-inspect` or any other diagnostic tool to pre-check availability; just call the tool and observe whether it succeeds or returns an error
Copy link

Copilot AI Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serena-activate_project likely doesn’t take a path argument. In the Serena MCP server tests, project activation is called with relative_path (e.g., {"relative_path": "go_project"}), so this instruction may cause the smoke workflow to fail when the agent follows it. Update the instruction to use the correct argument name (likely relative_path, probably set to "." / repo root) or omit the argument if the tool defaults to the configured project root.

Suggested change
- Call the `serena-activate_project` tool DIRECTLY with `path: "${{ github.workspace }}"` — do NOT use `mcp-inspect` or any other diagnostic tool to pre-check availability; just call the tool and observe whether it succeeds or returns an error
- Call the `serena-activate_project` tool DIRECTLY with `relative_path: "."` — do NOT use `mcp-inspect` or any other diagnostic tool to pre-check availability; just call the tool and observe whether it succeeds or returns an error

Copilot uses AI. Check for mistakes.
- A failure of `mcp-inspect` or any other indirect check does NOT indicate that Serena tools are unavailable; only report Serena as unavailable if the direct `serena-activate_project` call itself returns an error
Comment on lines +91 to +92
Copy link

Copilot AI Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This text forbids mcp-inspect, but the failure described in the PR comes from the prefixed tool name (agenticworkflows-mcp-inspect) used in routed mode. Consider explicitly calling out agenticworkflows-mcp-inspect here as well, otherwise the agent may still try that “diagnostic” pre-check.

Suggested change
- Call the `serena-activate_project` tool DIRECTLY with `path: "${{ github.workspace }}"` — do NOT use `mcp-inspect` or any other diagnostic tool to pre-check availability; just call the tool and observe whether it succeeds or returns an error
- A failure of `mcp-inspect` or any other indirect check does NOT indicate that Serena tools are unavailable; only report Serena as unavailable if the direct `serena-activate_project` call itself returns an error
- Call the `serena-activate_project` tool DIRECTLY with `path: "${{ github.workspace }}"` — do NOT use `mcp-inspect`, `agenticworkflows-mcp-inspect`, or any other diagnostic tool to pre-check availability; just call the tool and observe whether it succeeds or returns an error
- A failure of `mcp-inspect`, `agenticworkflows-mcp-inspect`, or any other indirect check does NOT indicate that Serena tools are unavailable; only report Serena as unavailable if the direct `serena-activate_project` call itself returns an error

Copilot uses AI. Check for mistakes.
- After activation, call the `serena-find_symbol` tool to search for symbols and verify that at least 3 symbols are found in the results
4. **Make Build Testing**: Use the `safeinputs-make` tool to build the project (use args: "build") and verify it succeeds
5. **Playwright Testing**: Use the playwright tools to navigate to https://github.com and verify the page title contains "GitHub" (do NOT try to install playwright - use the provided MCP tools)
6. **File Writing Testing**: Create a test file `/tmp/gh-aw/agent/smoke-test-copilot-${{ github.run_id }}.txt` with content "Smoke test passed for Copilot at $(date)" (create the directory if it doesn't exist)
Expand Down