Repository navigation
Run the tests solution wide, locally and in CI - #1119
Merged
Merged
Conversation
A solution wide dotnet test ran EfLocalDb.ReportTrxHelper as a test project, found zero tests, and failed the whole run with exit code 8 even when every test passed. The helper already sets IsTestProject=false, but under Microsoft.Testing.Platform dotnet test goes by IsTestingPlatformApplication, which NUnit3TestAdapter's targets assign from EnableNUnitRunner unconditionally. Set it to false in a Directory.Build.targets beside the project, which is imported after package targets and so is not overwritten. The entry point and extension registration default to that property, so they are now turned on explicitly and the helper stays the same testing platform executable.
One dotnet test over the solution replaces the loop that ran each project in turn and deleted every LocalDB instance in between. It runs the projects in parallel, picks up new test projects without a list to maintain, and is what a developer runs locally.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two commits: a solution wide
dotnet testnow passes, and CI uses it.CI. The Test step is one command in place of the loop that ran seven projects in turn and deleted every LocalDB instance in between:
LoadTestis now part of the CI run; it was not in the old list. Its 100 tests are explicit, so they are skipped and it reports as passed.Helper.
EfLocalDb.ReportTrxHelperis no longer run bydotnet test, which is what made a solution wide run fail.src/EfLocalDb.ReportTrxHelper/Directory.Build.targetssetsIsTestingPlatformApplicationtofalse.EfLocalDb.ReportTrxHelper.csprojsetsGenerateTestingPlatformEntryPointandGenerateSelfRegisteredExtensionstotrueexplicitly, since both default to that property. The helper is still the same testing platform executable, with the generatedMainand the TRX extension registered.Why
A solution wide
dotnet test --solution src/LocalDb.slnxfailed even when every test passed:The helper hosts no tests.
ReportTrxGuardTestslaunches it to prove the guard, and its project file already saysIsTestProject=false. Under Microsoft.Testing.Platform that is not whatdotnet testlooks at: it runs any project whereIsTestingPlatformApplicationistrue, andNUnit3TestAdapter.targetsassigns that fromEnableNUnitRunnerunconditionally.That is also why the override is in a
Directory.Build.targetsrather than in the project file. Package targets are imported after the project file's own properties, so a value set there is overwritten.Directory.Build.targetsis imported after the package targets.CI never hit this, since it ran a fixed list of projects one at a time. With the helper excluded it can run the solution, as a developer does.
Verified
dotnet test, Debug, before: exit code 8, "Failed!", as above.dotnet test, Debug, after: exit code 0, "Passed!", 469 passed, 0 failed, 101 skipped, and the helper is not listed.--report-trxstill exits non zero with the guard's message.ReportTrxGuardTestspass (7 of 7) in Debug and in Release, which is what CI builds.falsein both configurations.Not run: the full suite in Release. CI will do that here.
Alternative considered
The guard only reads the command line and the call stack, so the helper could be a plain console app with no NUnit or TRX packages, and would then never be mistaken for a test project. I kept it as a real test host because that looked deliberate.
Correction to #1118
That PR's description said a solution wide
dotnet testfailed locally with instance contention errors (Could not obtain exclusive lock on database 'model', dropped pipes), and that I had not checked whethermaindid the same. It does not. The failure was two of my own solution wide runs overlapping on the same LocalDB instances: the first had been cut short on screen by aSelect-Object -Firstin the pipeline but kept running in the background. Running one project twice at once reproduces those errors; a single clean run passes.