Disable net9.0 integration tests on net10 branch #33775
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Copilot
AI
changed the title
[WIP] Disable net9.0 tests on net10 branch
Disable net9.0 integration tests on net10 branch
Jan 29, 2026
Member
|
@copilot can you rebase this on net10.0 branch? |
Contributor
Author
Done! Successfully rebased onto net10.0 (41c168f). The branch now has the latest changes from net10.0 with my 2 commits on top. The rebase involved skipping 2 intermediate commits that had merge conflicts but were not part of this PR's changes. All test file modifications remain intact. Note: Due to the rebase, the branch history has diverged from origin. A force push is needed to update the remote branch. |
Copilot stopped work on behalf of
rmarinho due to an error
January 29, 2026 17:58
> [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## Summary Refactors the test-writing infrastructure from a monolithic agent to a dispatcher + skills architecture for better maintainability and extensibility. ## Changes ### Skills - **Renamed** `write-tests` → `write-ui-tests` (clearer naming) - **Added** `write-xaml-tests` skill for XAML unit tests (credit: @StephaneDelcroix) ### Agents - **Renamed** `uitest-coding-agent` → `write-tests-agent` - **Simplified** from 635 lines to 79 lines (dispatcher pattern) - Agent now determines test type and invokes appropriate skill ### References Updated - README-AI.md - copilot-instructions.md - pr.md - sandbox.instructions.md - agents.instructions.md - pr-finalize/SKILL.md ## Architecture | Before | After | |--------|-------| | `uitest-coding-agent.md` (635 lines) | `write-tests-agent.md` (79 lines) - dispatcher | | `write-tests/SKILL.md` | `write-ui-tests/SKILL.md` - UI tests | | — | `write-xaml-tests/SKILL.md` - XAML unit tests | ## Future Work - Wire `write-xaml-tests` into `write-tests-agent` dispatch table - Add additional test type skills (unit tests, integration tests) --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
### Description of Change This pull request introduces a new developer skill for running .NET MAUI device tests locally on iOS simulators. It adds comprehensive documentation and two PowerShell scripts to streamline building, running, and managing device tests using xharness and the .NET SDK. The scripts automate simulator selection, build configuration, and test execution, making it easier for developers to verify iOS compatibility and run targeted tests. **Skill documentation and workflow:** - Added `.github/skills/run-device-tests/SKILL.md` with detailed instructions, prerequisites, usage scenarios, supported projects, example commands, and workflow for running device tests on iOS simulators using xharness and PowerShell. **Device test automation scripts:** - Added `Run-DeviceTests.ps1` script to build and run specified MAUI device test projects on iOS simulators, supporting options for iOS version selection, test filtering, build-only mode, and output directory customization. The script also summarizes test results and failed tests. - Added `List-Simulators.ps1` script to enumerate available iOS simulators, with support for filtering by iOS version and showing only booted simulators, helping users select appropriate test targets. --------- Co-authored-by: Shane Neuville <shneuvil@microsoft.com> Co-authored-by: Jakub Florkowski <kubaflo123@gmail.com>
> [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ### Description of Change Adds a new `ai-summary-comment` skill for posting automated progress comments on GitHub PRs. **New files added:** - `.github/skills/ai-summary-comment/SKILL.md` - Skill definition and usage guide - `.github/skills/ai-summary-comment/IMPROVEMENTS.md` - Documentation of dynamic section extraction improvements - `.github/skills/ai-summary-comment/NO-EXTERNAL-REFERENCES-RULE.md` - Self-contained comment requirements - `.github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1` - Main script for posting aggregated PR comments - `.github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1` - PR finalize comment script - `.github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1` - Try-fix phase comment script - `.github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1` - Test verification comment script - `.github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1` - Write tests comment script **Key features:** - Dynamic section extraction from PR state files - Flexible regex pattern matching for section titles - Two-level validation (errors vs warnings) - Debug mode for troubleshooting - Creates single aggregated review comment with collapsible sections per commit
…agnostics (#33705) <!-- Please let the below note in for people that find this PR --> > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## Description This PR addresses multiple sources of UI test flakiness identified from analyzing the last 10 main branch UITest CI runs. ### 1. Screenshot Timing Fixes (`retryTimeout` parameter) Many visual tests were failing intermittently because screenshots were taken before animations/transitions completed. Added `retryTimeout: TimeSpan.FromSeconds(2)` to tests that tap buttons and immediately verify screenshots: **FeatureMatrix tests (39 files):** - All BoxView, Border, Brush, Button, CarouselView, CollectionView, ContentView, Frame, GraphicsView, Image, ImageButton, Label, ScrollView, Shadow, Shapes, Stepper, SwipeView, Switch, WebView tests **Issue tests:** - Issue10563 (SwipeView open/close) - Issue27730 (Shadow update when clipping) - Issue26662 (Dynamic FontImageSource color) ### 2. Instrumentation Crash Recovery Added detection and recovery for Android instrumentation crashes that were causing entire test fixtures to fail: - Added `IsInstrumentationCrash()` method with 12 crash signatures (socket hang up, ECONNRESET, session terminated, etc.) - Added recovery in `FixtureSetup` and `TestSetup` that calls `base.Reset()` to recreate the Appium driver session - This allows tests to recover from transient instrumentation failures instead of failing the entire fixture ### 3. OneTimeSetUp Diagnostic Attachment Fix Fixed an issue where diagnostic files (logcat, screenshots) attached during `OneTimeSetUp` failures were not visible in Azure DevOps test results: - NUnit limitation: `TestContext.AddTestAttachment` in `OneTimeSetUp` attaches to fixture context, not individual tests - Azure DevOps only displays per-test attachments - Solution: Store diagnostic file paths during fixture setup failures, then re-attach them to each individual test in `TearDown` ## Changes ### Infrastructure (`src/TestUtils/src/UITest.NUnit/UITestBase.cs`) - Added `_fixtureSetupDiagnosticFiles` list and `_fixtureSetupFailed` flag - Modified `TearDown` to re-attach fixture diagnostic files to each test - Added `storeForReattachment` parameter to `SaveDeviceDiagnosticInfo` and `SaveUIDiagnosticInfo` ### MAUI Test Base (`src/Controls/tests/TestCases.Shared.Tests/UITest.cs`) - Added `IsInstrumentationCrash()` detection with 12 crash signatures - Added crash recovery in `FixtureSetup` and `TestSetup` ### Test Files - 39 FeatureMatrix test files: Added `retryTimeout` - Issue10563.cs: Added `retryTimeout` to SwipeView screenshot tests - Issue27730.cs: Added `retryTimeout` to shadow update test - Issue26662.cs: Added `retryTimeout` to dynamic color test ## Testing Validated through multiple CI runs: - Build 1270034: iOS/macOS all passed, Android CoreClr passed after retry - Build 1270368: Identified additional timing issues in Issue27730 and Issue26662 - Build 1270512: In progress with latest fixes
# Conflicts: # eng/pipelines/common/provision.yml # Conflicts: # eng/pipelines/common/provision.yml
Adds automatic PR label management to the verify-tests-fail-without-fix skill. After test verification completes, the skill now updates labels on the PR to indicate verification status. **New Labels:** - s/ai-reproduction-confirmed - Tests correctly FAIL without fix - s/ai-reproduction-failed - Tests PASS without fix **Files Changed:** - SKILL.md - Added PR Labels documentation section - verify-tests-fail.ps1 - Added Update-VerificationLabels function --------- Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
This reverts commit d7d5e04.
Co-authored-by: rmarinho <1235097+rmarinho@users.noreply.github.com>
rmarinho
force-pushed
the
copilot/disable-net9-tests
branch
from
48b11e2 to
f601687
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Disable net9.0 tests on net10 branch
Problem
Completed
DotNetPrevious(net9.0) test cases in integration test filesChanges Summary
Files Modified:
WindowsTemplateTest.cs- 6 net9.0 test cases commented outAndroidTemplateTests.cs- 4 net9.0 test cases commented outSimpleTemplateTest.cs- 12 net9.0 test cases commented out + entire InstallPackagesIntoUnsupportedTfmFails methodTotal: 26 test cases disabled
All include TODO comments:
// TODO: Re-enable net9.0 tests - see https://github.com/dotnet/maui/issues/XXXXXRebase
Successfully rebased onto latest net10.0 (41c168f). Skipped 2 intermediate commits with conflicts that weren't part of this PR.
Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.