Skip to content

Support outcome filtering option in test results - #1011

Merged
Manoj Singaperumal (skmanoj) merged 5 commits into
microsoft:mainfrom
vinayakmsft:users/vinayakjoshi/filterbyoutcometestplansfromforkedrepo
Mar 18, 2026
Merged

Manoj Singaperumal (skmanoj) merged 5 commits into
microsoft:mainfrom
vinayakmsft:users/vinayakjoshi/filterbyoutcometestplansfromforkedrepo

Conversation

@vinayakmsft

@vinayakmsft Vinayak (vinayakmsft) commented Mar 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary
Returns useful fields from test results, including testCaseTitle, errorMessage, stackTrace.
Adds optional outcomes parameter for server-side filtering (e.g. ["Failed", "Aborted"])
Why switch from getTestResultDetailsForBuild to getTestRuns + getTestResults
The previous API (getTestResultDetailsForBuild) returns lightweight result entries even with shouldIncludeResults=true — they contain only id, outcome, durationInMs, and testCaseReferenceId, but not testCaseTitle, errorMessage, or stackTrace. Its $filter parameter supports OData-style outcome filtering, but the results lack the detail fields that make them actionable.

getTestResults (per run) returns full TestCaseResult objects with all fields populated, and natively supports an outcomes parameter for server-side filtering. This avoids fetching all results when only failed test cases are needed.

GitHub issue number 887 and 985

Before the changes

image

After the changes - focused only on the failed test by using the filter logic

image

✅ PR Checklist

  • I have read the contribution guidelines
  • I have read the code of conduct guidelines
  • Title of the pull request is clear and informative.
  • 👌 Code hygiene
  • 🔭 Telemetry added, updated, or N/A
  • 📄 Documentation added, updated, or N/A
  • 🛡️ Automated tests added, or N/A

🧪 How did you test it?

Local mcp testing

Replace with use cases tested and models used

@vinayakmsft
Vinayak (vinayakmsft) requested a review from a team as a code owner March 16, 2026 06:38
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Mar 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@1d5f0eb). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/tools/test-plans.ts 77.77% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1011   +/-   ##
=======================================
  Coverage        ?   93.78%           
=======================================
  Files           ?       16           
  Lines           ?     1626           
  Branches        ?      344           
=======================================
  Hits            ?     1525           
  Misses          ?       23           
  Partials        ?       78           
Flag Coverage Δ
unittests 93.78% <77.77%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread src/tools/test-plans.ts Outdated
Comment thread src/tools/test-plans.ts Outdated
Comment thread src/tools/test-plans.ts Outdated
Comment thread src/tools/test-plans.ts Outdated
Comment thread src/tools/test-plans.ts
@lizy14

Copy link
Copy Markdown

nit: update PR title to better reflect the scope of changes. It's not just support output filtering; it also added test case name, error message, stack trace etc. in response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants