Skip to content

[ci] Test retries don't clean between runs #8687

Open
@jpobst

Description

For the tests that use Azure Pipelines' built-in retryCountOnTaskFailure (like Mono.Android.NET-Tests), the test suite is re-run immediately after the failure. We cannot inject any extra steps before the retry. This means that the second run is an incremental build because the obj/bin directories are not cleaned. Additionally, the logs from the first run are not captured.

This could lead to passing of tests that fail on initial builds and only succeed on incremental builds.

Given the inflexibility of retryCountOnTaskFailure, we likely need to turn it off completely.

We should also examine our usage of dotnet-test-slicer to see if it suffers from the same issue. At least with dotnet-test-slicer we can run steps between the initial run and the retry, so we can add manual cleans if needed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Area: Unit TestsIssues with repo unit tests.bugComponent does not function as intended.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions