Include artifact paths in analyzed failed pipeline test results - #16714
Merged
Ben Broderick Phillips (benbp) merged 4 commits intoAug 12, 2026
Merged
Conversation
ReilleyMilne (ReilleyMilne)
commented
Aug 12, 2026
Member
- Updates pipeline analysis results to include artifact file paths, allowing other tools to fetch failed test details directly.
- Updates an out-of-date reference to the azsdk analyze tool in the azure-sdk-qa-bot-agent.
…flow. Also updating an out of date reference to the analyze tool in the sdk-ai-bot.
…reilleymilne/analyze-failed-tests-by-file
…reilleymilne/analyze-failed-tests-by-file
ReilleyMilne (ReilleyMilne)
requested review from
a team,
Jiaqi Zhang (JiaqiZhang-Dev),
Crystal YU (chunyu3) and
Renhe Li (lirenhe)
as code owners
August 12, 2026 21:29
Copilot started reviewing on behalf of
ReilleyMilne (ReilleyMilne)
August 12, 2026 21:30
View session
|
Azure Pipelines: Successfully started running 3 pipeline(s). 59 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the azsdk_analyze_pipeline analysis output to include per-artifact file paths for failed test results, enabling follow-on tools to fetch detailed failure data directly. It also refreshes pipeline-analysis guidance in the QA bot agent and pipeline-analysis skill documentation.
Changes:
- Change
failed_pipeline_testsfrom a platform-keyed dictionary to a list of per-artifact entries (artifact_file_path,platform,failed_test_titles). - Update pipeline artifact parsing to emit one result per artifact file, including the file path.
- Refresh pipeline-analysis documentation (agent instruction + skill) to describe the new output and the related “fetch details” tools.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/sdk-ai-bots/azure-sdk-qa-bot-agent/agents/chat_agent/instruction.md | Updates the QA bot agent guidance for Azure DevOps pipeline analysis usage. |
| tools/azsdk-cli/Azure.Sdk.Tools.Mock/Handlers/Pipeline/AnalyzeAndArtifactsHandlers.cs | Updates the mock azsdk_analyze_pipeline handler to emit per-artifact failed test entries with file paths. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli/Models/Pipeline/AzurePipelineAnalysis.cs | Updates the analysis model to use List<FailedTestArtifact> and introduces the FailedTestArtifact type. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli/Helpers/Pipeline/PipelineAnalysisHelper.cs | Updates artifact parsing to produce per-file failed test entries including artifact_file_path. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Helpers/Pipeline/PipelineAnalysisHelperTests.cs | Updates unit tests to validate the new per-artifact failed test shape. |
| .github/skills/azsdk-common-pipeline-analysis/SKILL.md | Updates the pipeline-analysis skill to document the new failed_pipeline_tests format and how to fetch failure details. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Ben Broderick Phillips (benbp)
approved these changes
Aug 12, 2026
This was referenced Aug 12, 2026
Merged
Merged
Merged
Ben Broderick Phillips (benbp)
enabled auto-merge (squash)
August 12, 2026 22:59
azure-sdk-automation Bot
added a commit
to Azure/azure-sdk-for-js
that referenced
this pull request
Aug 12, 2026
Sync .github/skills directory with azure-sdk-tools for PR Azure/azure-sdk-tools#16714 See [eng/common workflow](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/README.md#workflow) --------- Co-authored-by: ReilleyMilne <reilleymilne@gmail.com>
Ben Broderick Phillips (benbp)
disabled auto-merge
August 12, 2026 23:14
Ben Broderick Phillips (benbp)
deleted the
reilleymilne/analyze-failed-tests-by-file
branch
August 12, 2026 23:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.