Repository navigation
testplan_show_test_results_from_build_id still omits detailed fields after #1011 #1682
Description
Activity
Have you tried using the remote MCP Server instead?
Contract-level bisect, in case it helps narrow the window. I pin tool contracts per release and diff hashes (method at the bottom). For
testplan_show_test_results_from_build_id:version contract hash tools pinned 2.5.0 f5be27eb260e60b186 2.6.0 a3440170985a743d90 2.7.0 a3440170985a743d90 2.8.0 a3440170985a743d90 2.9.0 a3440170985a743d40 2.10.0 a3440170985a743d40 Two takeaways:
- This tool's contract changed exactly once, 2.5.0 -> 2.6.0, and never after. Support outcome filtering option in test results #1011 ("Support outcome filtering option in test results") merged 2026-03-18 - the same day 2.6.0 shipped. If the review commits that switched the implementation back to the lightweight fetch are the regression, they rode 2.6.0, and 2.6.0 is the version boundary to diff against. The 2.9.0 consolidation (90 -> 40 tools) never touched this tool's contract.
- The description never changed across the whole range - the promised fields have been advertised continuously while the serialized output dropped them, so nothing at the contract surface flags the regression. That's exactly the gap contract pinning can't see through either; response-shape assertions would be the next layer.
Method: pin-and-diff with an open-source zero-dependency tool, 4-command repro in the audit report: https://github.com/Paraphern/rugsnare/blob/main/audits/npm-top-mcp-drift-2026-10.md (Section 1 also has the full 2.5.0 -> 2.10.0 map: 75 removed / 29 added / 6 rewritten tool names - happy to post the lists here or as JSON if a migration note would use them).
- linked a pull request that will close this issuefix: update test results fetching logic to support outcome validation… #1684
on Oct 7, 2026 Please take a look at #1684 and let me know if this solves your issues. I would love to have more than myself testing it before we merge
Tested the branch at the contract level (pinned the built server and diffed against the 2.10.0 release). What I can confirm from here:
- Startup/tool surface: clean. 44 tools pinned vs 40 in the 2.10.0 release - the 4 extra come from main being ahead (
repo_pull_request_org,repo_file_write,wit_work_item_attachment_upload,wit_work_item_attachment_link), nothing removed. testplan_show_test_results_from_build_idcontract: description unchanged; input schema has exactly one change -outcomeswent from free-form string array to an enum of the 15 valid outcome values (Unspecified...NotImpacted). That matches the PR's "stricter schema validation for outcome filters" precisely.- What I can't verify without a real org + published test runs: the response fields themselves (
testCaseTitle,errorMessage,stackTrace). The paging-over-runs approach in the code is exactly what the testplan_show_test_results_from_build_id still omits detailed fields after #1011 #1682 root-cause analysis pointed at (per-run results return fullTestCaseResultobjects), so it looks right - but the runtime confirmation needs someone with a live instance.
One small note worth a changelog line when this ships: the
outcomesenum is technically a client-facing input-schema change - free-form strings (e.g. a lowercasefailed) that 2.10.0 accepted now fail validation. Probably exactly what you want, but agents/skills passing lowercase values will notice.For what it's worth, this package is on our weekly drift watch (pinned baselines, diffed on every release) - once the fix lands in a published version, the watcher will pick it up automatically.
- Startup/tool surface: clean. 44 tools pinned vs 40 in the 2.10.0 release - the 4 extra come from main being ahead (
Confirmed with a live organization and published test runs.
Baseline
@azure-devops/mcp2.10.0 returned onlyid,outcome,durationInMs, andrunIdfor the reproducing failed result. I built PR #1684 atc6357be, ran the full suite (1,351/1,351 passed), tool validation, and the TypeScript build, then invoked that exact branch against the same build/result. The PR returnedtestCaseTitle, the fullerrorMessage,automatedTestName, andautomatedTestStorage, so it resolves #1682 for this live case. Outcome filtering withFailedalso worked correctly.I also tested the hosted remote MCP server directly. It already returns those detailed fields for the same result. That server is different from my current setup: I am currently using the local stdio package (
npx -y @azure-devops/mcp@latest, currently 2.10.0), while the remote server is the Azure DevOps-hosted streamable-HTTP service and receives updates separately/earlier.That completes the stack nicely - your live-org run covers exactly the gap we flagged (response fields need a real org; we could only verify the contract surface). Between the maintainer's suite, the contract-level check and your live verification, #1684 looks merge-ready from the outside.
One observation worth a docs line for anyone pinning this package: the local/remote split you describe is itself a drift surface. The npm stdio channel (2.10.0, fields missing) and the hosted streamable-HTTP channel (fields present) are versioned separately - same product, different contracts per channel, and the remote one moves out-of-band from npm releases. Consumers who pin and diff only the npm package get no signal when the remote contract changes; worth stating explicitly whenever the remote server's versioning gets documented.
Summary
testplan_show_test_results_from_build_idstill returns lightweight test-result rows withouttestCaseTitle,errorMessage,stackTrace,automatedTestName, orautomatedTestStoragein@azure-devops/mcp2.10.0 and currentmain.This appears to be a regression introduced during review of #1011, which closed #887 and #985.
Current behavior
For a build with published test results, the tool returns rows shaped like:
{ "id": 100000, "outcome": "Passed", "durationInMs": 6753386, "runId": "1706006238" }The tool description promises test titles, error messages, and stack traces, but those properties are absent from the serialized response.
Root cause analysis
The first commit in #1011 changed the implementation from
getTestResultDetailsForBuildto:getTestRuns(project, buildUri)getTestResults(project, runId, ..., outcomes)for each runThat matches the PR description, which states that
getTestResultDetailsForBuildreturns lightweight entries without the detailed fields, while per-rungetTestResultsreturns fullTestCaseResultobjects.A later review commit in the same PR switched the implementation back to:
Current
mainand published 2.10.0 still use that API. The formatter then reads fields such asr.testCaseTitle, but they areundefinedon lightweight rows, soJSON.stringifysilently omits them.There was no later revert after merge: merge commit
67b9aa68169f90fc311b648ad470d5f95c2724b8remains an ancestor ofmain. The ineffective API choice was included in the final merged form of #1011.Test gap
The unit tests mock
getTestResultDetailsForBuildas returning fully populated result objects. That does not match the observed production API response, so the tests validate the formatter but not the API contract that supplies its input.Expected behavior
Each returned row should include, when available:
testCaseTitleerrorMessagestackTraceautomatedTestNameautomatedTestStorageoutcome, duration, result ID, and run IDOutcome filtering should continue to work without requiring callers to retrieve every passed result.
Suggested fix
Restore the per-run detailed-results path from the first revision of #1011:
getTestResultsfor each run with server-side outcome filtering.testCaseTitle,errorMessage, andstackTrace; do not mock those fields onto the lightweight response type.References