Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Feb 1, 2026

The GitHub MCP toolsets permissions mapping referenced two tools that don't exist in the remote server, causing the compiler's tool catalog to diverge from runtime capabilities.

Changes

Removed from pkg/workflow/data/github_toolsets_permissions.json:

  • repos toolset: get_repository (never implemented in MCP server)
  • code_security toolset: create_code_scanning_alert (read-only toolset, write operation unavailable)

Updated test fixtures:
Replaced get_repository references with get_file_contents in:

  • copilot_engine_test.go
  • copilot_mcp_http_integration_test.go
  • cache_memory_threat_detection_test.go
  • cache_memory_import_test.go
  • compiler_cache_test.go
  • github_toolset_validation_integration_test.go

The repos toolset now accurately lists 10 available tools. The code_security toolset correctly reflects its read-only nature with 2 tools.

Original prompt

This section details on the original issue you should resolve

<issue_title>Remove non-existent tools from GitHub MCP toolsets mapping</issue_title>
<issue_description>### Summary

This PR updates the GitHub MCP toolsets permissions mapping to accurately reflect the tools actually available in the GitHub MCP remote server.

Changes

Removed two tools from pkg/workflow/data/github_toolsets_permissions.json that are listed in the JSON mapping but don't actually exist in the GitHub MCP server:

  1. repos toolset: Removed get_repository

    • This tool is not available in the MCP server
    • The repos toolset provides other repository operations like get_file_contents, list_commits, etc.
  2. code_security toolset: Removed create_code_scanning_alert

    • This write operation is not available in the current MCP server
    • The toolset still provides read operations: get_code_scanning_alert and list_code_scanning_alerts

Verification

The changes were verified by comparing the JSON mapping against the actual tools discovered in the GitHub MCP remote server during the comprehensive tools audit run on January 6, 2026. The audit systematically explored all 19 toolsets and cataloged 56 available tools.

Impact

This ensures that:

  • The compiler's understanding of available GitHub MCP tools matches reality
  • Workflows won't attempt to use non-existent tools
  • The toolsets documentation accurately reflects actual capabilities

Testing

  • Verified tools list against MCP server capabilities from previous audit
  • Confirmed JSON structure and formatting preserved
  • Ensured alphabetical ordering maintained within toolsets

AI generated by GitHub MCP Remote Server Tools Report Generator


[!NOTE]
This was originally intended as a pull request, but the git push operation failed.

Workflow Run: View run details and download patch artifact

The patch file is available in the agent-artifacts artifact in the workflow run linked above.

To apply the patch locally:

# Download the artifact from the workflow run https://github.com/githubnext/gh-aw/actions/runs/21563453541
# (Use GitHub MCP tools if gh CLI is not available)
gh run download 21563453541 -n agent-artifacts

# The patch file will be at agent-artifacts/tmp/gh-aw/aw.patch after download
# Apply the patch
git am agent-artifacts/tmp/gh-aw/aw.patch
Show patch preview (43 of 43 lines)
From bfd0dd46836747b227b5279e6773627b65e3386d Mon Sep 17 00:00:00 2001
From: Claude Code <claude-code@anthropic.com>
Date: Sun, 1 Feb 2026 13:13:46 +0000
Subject: [PATCH] Remove non-existent tools from GitHub MCP toolsets mapping

Removed tools that are listed in the JSON mapping but don't actually exist in the GitHub MCP server:

- repos: Removed 'get_repository' (not available in MCP server)
- code_security: Removed 'create_code_scanning_alert' (not available in MCP server)

This ensures the toolsets mapping accurately reflects the actual capabilities of the GitHub MCP remote server.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---
 pkg/workflow/data/github_toolsets_permissions.json | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/pkg/workflow/data/github_toolsets_permissions.json b/pkg/workflow/data/github_toolsets_permissions.json
index e53534e..ecc634a 100644
--- a/pkg/workflow/data/github_toolsets_permissions.json
+++ b/pkg/workflow/data/github_toolsets_permissions.json
@@ -12,7 +12,7 @@
       "description": "Repository operations",
       "read_permissions": ["contents"],
       "write_permissions": ["contents"],
-      "tools": ["get_commit", "get_file_contents", "get_latest_release", "get_release_by_tag", "get_repository", "get_repository_tree", "get_tag", "list_branches", "list_commits", "list_releases", "list_tags"]
+      "tools": ["get_commit", "get_file_contents", "get_latest_release", "get_release_by_tag", "get_repository_tree", "get_tag", "list_branches", "list_commits", "list_releases", "list_tags"]
     },
     "issues": {
       "description": "Issue management",
@@ -36,7 +36,7 @@
       "description": "Code scanning alerts",
       "read_permissions": ["security-events"],
       "write_permissions": ["security-events"],
-      "tools": ["get_code_scanning_alert", "list_code_scanning_alerts", "create_code_scanning_alert"]
+      "tools": ["get_code_scanning_alert", "list_code_scanning_alerts"]
     },
     "de
... (truncated)

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- Remove get_repository from repos toolset (tool doesn't exist in MCP server)
- Remove create_code_scanning_alert from code_security toolset (tool doesn't exist in MCP server)
- Update test files to use get_file_contents instead of get_repository
- Rebuild binary to embed updated JSON

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

pelikhan commented Feb 1, 2026

@copilot review changes and compare with readme of GitHub-MCP-server repo

Copilot AI changed the title [WIP] Remove non-existent tools from GitHub MCP toolsets mapping Remove non-existent tools from GitHub MCP toolsets mapping Feb 1, 2026
Copilot AI requested a review from pelikhan February 1, 2026 23:19
Copy link
Contributor Author

Copilot AI commented Feb 1, 2026

@copilot review changes and compare with readme of GitHub-MCP-server repo

Verified against the official GitHub MCP Server README. The removals are correct:

get_repository: Not listed in the repos toolset tools. The repos toolset includes tools like get_file_contents, get_commit, list_branches, but no get_repository.

create_code_scanning_alert: Not listed in the code_security toolset tools. The toolset only has read operations: get_code_scanning_alert and list_code_scanning_alerts.

The changes align with the actual MCP server capabilities documented in commit 5d8e075.

@pelikhan pelikhan marked this pull request as ready for review February 1, 2026 23:28
@pelikhan pelikhan merged commit 981d99a into main Feb 1, 2026
149 checks passed
@pelikhan pelikhan deleted the copilot/remove-non-existent-tools branch February 1, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove non-existent tools from GitHub MCP toolsets mapping

2 participants