Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 27, 2026

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Summary

Refactors the test-writing infrastructure from a monolithic agent to a dispatcher + skills architecture for better maintainability and extensibility.

Changes

Skills

  • Renamed write-testswrite-ui-tests (clearer naming)
  • Added write-xaml-tests skill for XAML unit tests (credit: @StephaneDelcroix)

Agents

  • Renamed uitest-coding-agentwrite-tests-agent
  • Simplified from 635 lines to 79 lines (dispatcher pattern)
  • Agent now determines test type and invokes appropriate skill

References Updated

  • README-AI.md
  • copilot-instructions.md
  • pr.md
  • sandbox.instructions.md
  • agents.instructions.md
  • pr-finalize/SKILL.md

Architecture

Before After
uitest-coding-agent.md (635 lines) write-tests-agent.md (79 lines) - dispatcher
write-tests/SKILL.md write-ui-tests/SKILL.md - UI tests
write-xaml-tests/SKILL.md - XAML unit tests

Future Work

  • Wire write-xaml-tests into write-tests-agent dispatch table
  • Add additional test type skills (unit tests, integration tests)

Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
Copilot AI changed the title [WIP] Update markdown files from PR 33705 Extract markdown documentation changes from PR #33705 Jan 27, 2026
Copilot AI requested a review from PureWeen January 27, 2026 13:01
- Rename write-tests skill to write-ui-tests
- Rename uitest-coding-agent to write-tests-agent (dispatcher)
- Add write-xaml-tests skill (credit: Stephane Delcroix)
- Update all references across README-AI.md, copilot-instructions.md, pr.md, etc.
- Fix pr-finalize to reference write-tests-agent instead of specific skill
@PureWeen PureWeen changed the title Extract markdown documentation changes from PR #33705 [XEXPR] Refactor test skills/agents to dispatcher pattern Jan 27, 2026
@PureWeen PureWeen marked this pull request as ready for review January 27, 2026 15:05
Copilot AI review requested due to automatic review settings January 27, 2026 15:05
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the repo’s Copilot test-writing setup from a single UI-test-focused agent/skill into a dispatcher agent (write-tests-agent) plus separate skills for UI tests and XAML unit tests.

Changes:

  • Adds write-tests-agent dispatcher and introduces write-ui-tests + write-xaml-tests skills (removing the old write-tests skill and uitest-coding-agent).
  • Updates documentation and guidance files to reference the new agent/skill names and usage patterns.
  • Expands UI test authoring guidance in .github/instructions/uitests.instructions.md.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
.github/skills/write-xaml-tests/SKILL.md New skill for creating XAML unit tests in Controls.Xaml.UnitTests.
.github/skills/write-ui-tests/SKILL.md New skill for creating UI tests and verifying they reproduce issues.
.github/skills/write-tests/SKILL.md Removes the old monolithic write-tests skill.
.github/skills/pr-finalize/SKILL.md Updates references away from old test-writing entry points.
.github/instructions/uitests.instructions.md Adds/updates UI test guidance (notably screenshot verification guidance).
.github/instructions/sandbox.instructions.md Updates “when not to use” guidance to point to write-tests-agent.
.github/instructions/agents.instructions.md Updates agent naming examples to write-tests-agent.
.github/copilot-instructions.md Updates agent/skill catalog and delegation examples for new architecture.
.github/agents/write-tests-agent.md Adds dispatcher agent that routes to UI vs XAML test-writing skills.
.github/agents/uitest-coding-agent.md Removes the old UI-test-specific agent definition.
.github/agents/pr.md Updates PR agent references to the new test-writing skill names.
.github/README-AI.md Updates the “how to use agents” README to reflect new agent/skills.

Comment on lines 62 to 66
1. Read the XAML unit test guidelines
2. Create XAML and code-behind files
3. Build and run the test

**🛑 CRITICAL:** The skill enforces that tests must FAIL before reporting success. A passing test does NOT prove it catches the bug.
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The agent text says the invoked skill “enforces that tests must FAIL before reporting success.” That’s true for write-ui-tests, but write-xaml-tests currently allows both reproduction tests (fail before fix) and regression tests (pass) and does not include the same hard requirement. Please reword this to apply only to UI tests, or update write-xaml-tests to explicitly enforce the same failure-first rule if that’s intended.

Copilot uses AI. Check for mistakes.
Comment on lines 205 to 207
2. **write-tests-agent** - Agent for writing tests. Determines test type and invokes appropriate skill (currently UI tests via write-ui-tests)
- **Use when**: Creating new tests for issues or PRs
- **Capabilities**: Test type determination, skill invocation, test verification
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description still says write-tests-agent currently supports only UI tests via write-ui-tests, but the PR adds write-xaml-tests and write-tests-agent.md claims XAML dispatch support. Please update this section to reflect current capabilities (UI + XAML) to avoid confusing users and the delegation logic.

Suggested change
2. **write-tests-agent** - Agent for writing tests. Determines test type and invokes appropriate skill (currently UI tests via write-ui-tests)
- **Use when**: Creating new tests for issues or PRs
- **Capabilities**: Test type determination, skill invocation, test verification
2. **write-tests-agent** - Agent for writing tests. Determines test type (UI vs XAML) and invokes the appropriate skill (`write-ui-tests`, `write-xaml-tests`)
- **Use when**: Creating new tests for issues or PRs
- **Capabilities**: Test type determination (UI and XAML), skill invocation, test verification

Copilot uses AI. Check for mistakes.
- Add retryTimeout parameter to VerifyScreenshot (from PR #33705)
- Fix regex to match both '.' and ',' decimal separators
- Update write-tests-agent to clarify XAML vs UI test behavior
- Update copilot-instructions to reflect XAML test support
@PureWeen
Copy link
Member

/azp run maui-pr-uitests, maui-pr-devicetests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@PureWeen PureWeen merged commit 3ff8868 into main Jan 27, 2026
112 of 163 checks passed
@PureWeen PureWeen deleted the copilot/update-markdown-files branch January 27, 2026 17:13
@kubaflo kubaflo added the area-ai-agents Copilot CLI agents, agent skills, AI-assisted development label Jan 28, 2026
rmarinho pushed a commit that referenced this pull request Jan 29, 2026
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

## Summary

Refactors the test-writing infrastructure from a monolithic agent to a
dispatcher + skills architecture for better maintainability and
extensibility.

## Changes

### Skills
- **Renamed** `write-tests` → `write-ui-tests` (clearer naming)
- **Added** `write-xaml-tests` skill for XAML unit tests (credit:
@StephaneDelcroix)

### Agents  
- **Renamed** `uitest-coding-agent` → `write-tests-agent`
- **Simplified** from 635 lines to 79 lines (dispatcher pattern)
- Agent now determines test type and invokes appropriate skill

### References Updated
- README-AI.md
- copilot-instructions.md
- pr.md
- sandbox.instructions.md
- agents.instructions.md
- pr-finalize/SKILL.md

## Architecture

| Before | After |
|--------|-------|
| `uitest-coding-agent.md` (635 lines) | `write-tests-agent.md` (79
lines) - dispatcher |
| `write-tests/SKILL.md` | `write-ui-tests/SKILL.md` - UI tests |
| — | `write-xaml-tests/SKILL.md` - XAML unit tests |

## Future Work

- Wire `write-xaml-tests` into `write-tests-agent` dispatch table
- Add additional test type skills (unit tests, integration tests)

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai-agents Copilot CLI agents, agent skills, AI-assisted development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants