Skip to content

Improve agent workflow guidance - #4777

Open
paulmedynski wants to merge 5 commits into
mainfrom
dev/automation/agent-improvements
Open

paulmedynski wants to merge 5 commits into
mainfrom
dev/automation/agent-improvements

Conversation

@paulmedynski

@paulmedynski paulmedynski commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Description

Separates repository agent-workflow and policy improvements from #4130:

  • Generalizes Azure DevOps work-item Markdown guidance and resource-access tooling.
  • Improves prompt guidance for generating prompts and triaging pipeline failures.
  • Adds repository Markdown lint configuration.
  • Clarifies coding-style guidance for line wrapping and regions.

This PR contains documentation, prompt, policy, and tooling configuration changes only. It does not change product APIs or runtime behavior.

Issues

Split from #4130 in response to review feedback requesting that unrelated agent and policy improvements be reviewed separately.

Testing

Not run. The changes affect Markdown guidance, prompts, policy documentation, and Markdown lint configuration only; no documentation test target is defined for these files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@paulmedynski
paulmedynski requested review from a team and a balanced review from Copilot October 2, 2026 11:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The prompt generator prescribes repository-root-relative skill links that standard VS Code prompt-file resolution cannot resolve.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Updates repository guidance for agent workflows, Markdown style, and Azure DevOps tooling.

Changes:

  • Generalizes Azure DevOps access guidance.
  • Adds Markdown lint configuration.
  • Clarifies prompt-link and coding-style guidance.
File Description
policy/​coding-style.md Clarifies wrapping and region guidance.
.markdownlint.jsonc Configures Markdown lint rules.
.github/​prompts/​triage-pipeline-failures.prompt.md Adds access preference ordering.
.github/​prompts/​generate-prompt.prompt.md Documents skill-link generation.
.github/​prompts/​audit-variable-groups.prompt.md Makes Azure DevOps access mechanism-agnostic.
.github/​instructions/​ado-work-items-markdown.instructions.md Generalizes work-item tooling guidance.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/prompts/generate-prompt.prompt.md
@paulmedynski paulmedynski added the Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems. label Oct 2, 2026
@paulmedynski paulmedynski added this to the 8.0.0-preview1 milestone Oct 2, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 11:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The audit can incorrectly mark groups unused by omitting active non-default pipeline branches.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread .github/prompts/audit-variable-groups.prompt.md Outdated
Addresses review feedback on #4777: derive branch scope from trigger
filters, recent run history, and template refs in addition to each
pipeline's default branch, and report groups as usage unknown when a
pipeline's branch scope cannot be determined.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 13:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The variable-group audit can falsely mark referenced groups as unused and fails to recognize its own default unused marker.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Match unused marker variants using the correct deletion substring

.github/​prompts/​audit-variable-groups.prompt.md:83

The default marker selected above is UNUSED - PENDING DELETION, but this detector requires the substring delete; deletion does not contain delete. On a later audit, a now-used group can therefore retain the stale unused marker because the replacement rule never recognizes it. Match the resolved marker exactly and use a delet stem only for legacy variants.

Comment thread .github/prompts/audit-variable-groups.prompt.md Outdated
Addresses review feedback on #4777:
- Derive scope from every branch and tag containing an enabled or paused
  pipeline's YAML, since manual, scheduled, and resource-triggered runs
  are not bounded by trigger filters or run history.
- Include paused YAML and Classic pipelines, and report unreferenced
  groups as usage unknown whenever the scope is restricted or incomplete.
- Recognize the resolved unused marker and "delet" variants so the
  default "PENDING DELETION" marker is detected on later audits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 14:08
@paulmedynski

Copy link
Copy Markdown
Contributor Author

Responses to review feedback that has no review thread:

Review body — Copilot (review 5392207841, "Previously missed")

  • .github/prompts/audit-variable-groups.prompt.md:83 — Match unused marker variants using the correct deletion substring. Fixed in e510609. A description is now treated as marked unused when it contains the resolved unused marker, or contains both "unused" and the stem "delet" (matching "delete", "deleted", and "deletion"), case-insensitively. The default UNUSED - PENDING DELETION marker is therefore recognized on later audits, so a now-used group's stale marker is replaced.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The variable-group audit can misclassify dormant YAML references as active and report usage on an unrelated default branch.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread .github/prompts/audit-variable-groups.prompt.md Outdated
Addresses review feedback on #4777:
- Start from each enabled or paused pipeline's YAML path and follow its
  templates transitively, so examples, fixtures, and disabled pipelines'
  YAML no longer count as usage.
- Resolve template paths, refs, and group values from fixed values or
  declared parameter values; otherwise mark the scope incomplete.
- Summarize long ref lists from refs where the group is referenced,
  never from an unrelated default branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 15:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The variable-group workflow can modify shared Azure DevOps resources and has not been validated end to end.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@paulmedynski
paulmedynski enabled auto-merge (squash) October 5, 2026 10:27
@paulmedynski paulmedynski added Hotfix 6.1.8 Hotfix 7.1.2 PRs targeting main that should be backported to release/7.1 for 7.1.2. labels Oct 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems. Hotfix 6.1.8 Hotfix 7.0.4 Hotfix 7.1.2 PRs targeting main that should be backported to release/7.1 for 7.1.2.

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

4 participants