Support filtering by area when running tool description evaluator

[srnagar]

What does this PR do?

This pull request adds support for filtering prompts by Azure service/tool prefix in the Tool Description Evaluator, making it easier to test prompts for specific services (like Key Vault or Storage). It introduces a new --area command line argument (and corresponding PowerShell script parameter) to filter prompts by tool name prefix, with automatic handling of common Azure service names. The documentation and help text have been updated to explain and demonstrate this new feature.

Tool Prefix Filtering Feature:

• Added support for a new --area command line argument in Program.cs and a corresponding Area parameter in Run-ToolDescriptionEvaluator.ps1 to filter prompts by tool name prefix (e.g., "keyvault" auto-expands to "azmcp_keyvault"). Multiple areas can be specified as a comma-separated list. [1] [2] [3] [4]
• Updated the prompt loading logic in LoadPromptsFromMarkdownAsync to filter prompts based on the specified area(s), with support for both auto-prefixed and explicitly prefixed tool names. Added user feedback if no prompts are found for the given area. [1] [2] [3]
• Enhanced help output and usage examples in both the CLI (ShowHelp in Program.cs) and PowerShell script to document the new filtering capability. [1] [2] [3]

Documentation Updates:

• Updated README.md to add a new section on "Tool Prefix Filtering Mode", provide detailed usage examples, and clarify how the filtering works with both the CLI and PowerShell script. [1] [2] [3]
• Expanded the command line options section and input data source examples in README.md to include the new --area option and its usage. [1] [2]

General Improvements:

• Minor refactoring to improve clarity in argument parsing and prompt filtering, and to provide clearer user feedback when no prompts match the filter. [1] [2]

These changes make it much easier to target and evaluate prompts for specific Azure services, improving the flexibility and usability of the Tool Description Evaluator.

GitHub issue number?

N/A

Pre-merge Checklist

• Required for All PRs
  • Read contribution guidelines
  • PR title clearly describes the change
  • Commit history is clean with descriptive messages (cleanup guide)
  • Added comprehensive tests for new/modified functionality
  • Updated servers/Azure.Mcp.Server/CHANGELOG.md and/or servers/Fabric.Mcp.Server/CHANGELOG.md for product changes (features, bug fixes, UI/UX, updated dependencies)
• For MCP tool changes:
  • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
  • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
  • Updated command list in /servers/Azure.Mcp.Server/docs/azmcp-commands.md and/or /docs/fabric-commands.md
  • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
  • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
• Extra steps for Azure MCP Server tool changes:
  • Updated test prompts in /servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
  • 👉 For Community (non-Microsoft team member) PRs:
    • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
    • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

[Copilot]

Pull Request Overview

This PR adds area filtering functionality to the Tool Description Evaluator, enabling users to focus on specific Azure service areas when running evaluations. This enhancement improves usability for both development and CI scenarios by allowing targeted testing of specific service areas.

• Added --area command-line parameter to filter prompts by Azure service area (e.g., "storage", "keyvault")
• Enhanced prompt loading logic to parse area headers and identifiers from markdown files
• Updated documentation, scripts, and test prompts to support the new filtering capability

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
eng/tools/ToolDescriptionEvaluator/Program.cs	Added area filtering parameter parsing and enhanced markdown loading logic to filter prompts by area
eng/tools/ToolDescriptionEvaluator/README.md	Updated documentation with area filtering usage examples and available area identifiers
eng/tools/ToolDescriptionEvaluator/Run-ToolDescriptionEvaluator.ps1	Added -Area parameter support to PowerShell script with updated examples
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md	Added area identifiers to each section for improved filtering support

[vcolin7]

The code itself looks good, but I think we should filter tools by using the command name prefixes instead of an arbitrary string in the markdown file, like azmcp_storage or azmcp_keyvault. This way we support both .md and .json files and don't need to add more info to either to achieve wat we want. We could also just ignore the azmcp_ prefix and match strings that include this --area/--filter right after. Moreso, we could allow to use this option more than once (or it being an array, whichever is easiest to implement) and allow for more flexible filtering: dotnet run --area "storage" --area "keyvault"