Skip to content

Include artifact paths in analyzed failed pipeline test results - #16714

Merged
Ben Broderick Phillips (benbp) merged 4 commits into
mainfrom
reilleymilne/analyze-failed-tests-by-file
Aug 12, 2026
Merged

Include artifact paths in analyzed failed pipeline test results#16714
Ben Broderick Phillips (benbp) merged 4 commits into
mainfrom
reilleymilne/analyze-failed-tests-by-file

Conversation

@ReilleyMilne

Copy link
Copy Markdown
Member
  • Updates pipeline analysis results to include artifact file paths, allowing other tools to fetch failed test details directly.
  • Updates an out-of-date reference to the azsdk analyze tool in the azure-sdk-qa-bot-agent.

…flow.

Also updating an out of date reference to the analyze tool in the sdk-ai-bot.
…reilleymilne/analyze-failed-tests-by-file
…reilleymilne/analyze-failed-tests-by-file
Copilot AI lite review requested due to automatic review settings August 12, 2026 21:29
@github-actions github-actions Bot added the azsdk-cli Issues related to Azure/azure-sdk-tools::tools/azsdk-cli label Aug 12, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
59 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the azsdk_analyze_pipeline analysis output to include per-artifact file paths for failed test results, enabling follow-on tools to fetch detailed failure data directly. It also refreshes pipeline-analysis guidance in the QA bot agent and pipeline-analysis skill documentation.

Changes:

  • Change failed_pipeline_tests from a platform-keyed dictionary to a list of per-artifact entries (artifact_file_path, platform, failed_test_titles).
  • Update pipeline artifact parsing to emit one result per artifact file, including the file path.
  • Refresh pipeline-analysis documentation (agent instruction + skill) to describe the new output and the related “fetch details” tools.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tools/sdk-ai-bots/azure-sdk-qa-bot-agent/agents/chat_agent/instruction.md Updates the QA bot agent guidance for Azure DevOps pipeline analysis usage.
tools/azsdk-cli/Azure.Sdk.Tools.Mock/Handlers/Pipeline/AnalyzeAndArtifactsHandlers.cs Updates the mock azsdk_analyze_pipeline handler to emit per-artifact failed test entries with file paths.
tools/azsdk-cli/Azure.Sdk.Tools.Cli/Models/Pipeline/AzurePipelineAnalysis.cs Updates the analysis model to use List<FailedTestArtifact> and introduces the FailedTestArtifact type.
tools/azsdk-cli/Azure.Sdk.Tools.Cli/Helpers/Pipeline/PipelineAnalysisHelper.cs Updates artifact parsing to produce per-file failed test entries including artifact_file_path.
tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Helpers/Pipeline/PipelineAnalysisHelperTests.cs Updates unit tests to validate the new per-artifact failed test shape.
.github/skills/azsdk-common-pipeline-analysis/SKILL.md Updates the pipeline-analysis skill to document the new failed_pipeline_tests format and how to fetch failure details.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/sdk-ai-bots/azure-sdk-qa-bot-agent/agents/chat_agent/instruction.md Outdated
azure-sdk-automation Bot added a commit to Azure/azure-sdk-for-js that referenced this pull request Aug 12, 2026
Sync .github/skills directory with azure-sdk-tools for PR
Azure/azure-sdk-tools#16714 See [eng/common
workflow](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/README.md#workflow)

---------

Co-authored-by: ReilleyMilne <reilleymilne@gmail.com>
@benbp
Ben Broderick Phillips (benbp) merged commit 1839d70 into main Aug 12, 2026
22 of 23 checks passed
@benbp
Ben Broderick Phillips (benbp) deleted the reilleymilne/analyze-failed-tests-by-file branch August 12, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

azsdk-cli Issues related to Azure/azure-sdk-tools::tools/azsdk-cli

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants