Excluding specific projects from dotnet test --solution

[afscrome]

How can I run dotnet test against a solution file, but skip a specific projects

With MSTest, we handled this by using dotnet test Foo.sln, and then

<IsTestProject Condition="'$(SkipDeployedTests)' == 'true'">false</IsTestProject>

At first glance, it looks like IsTestingPlatformApplication would be equivalent, but turning <IsTestingApplicationProject > off by itself is not enough as you need to change <OutputType> back to library to avoid compilation errors.

Even if that works, this is not ideal - we do actually want to run these tests, but at a later point in the pipeline. We could really benefit from MTP's capability to publishing the application as a test application, so we only need to run the already compiled application, rather than having to check out the source code, and build it before testing it. So really we do want to compile them as test applications, we just don't want dotnet test to run them at this point in CI. We can fudge around this if we also enable GenerateTestingPlatformEntryPoint and GenerateSelfRegisteredExtensions, but that feels rather messy, and also ends up fighting with nunit - see nunit/nunit3-vs-adapter#1378

<PropertyGroup Condition="'$(CI)' == 'true' OR '$(TF_BUILD)' == 'True'">
    <GenerateTestingPlatformEntryPoint>true</GenerateTestingPlatformEntryPoint>
    <GenerateSelfRegisteredExtensions>true</GenerateSelfRegisteredExtensions>
    <IsTestingPlatformApplication>false</IsTestingPlatformApplication>
</PropertyGroup>

Do you have any better suggestions for how we could handle this?

For context, our pipeline roughly does the following:

Job 1: Build & Test

1. dotnet restore foo.slnx
2. dotnet build foo.slnx
3. dotnet test --solution foo.slnx # We want this to run all tests except Foo.DeploymentTests
4. dotnet pack foo.slnx
5. dotnet publish foo.slnx # deployment tests set <IsPublishable> true so the test can be picked up
6. Publish artifacts.

Job 2: Deployment Validation

1. Deploy application
2. Run Foo.DeploymentTests.exe against the deployment (obtained from dotnet publish artifacts from previous stage) # the deployment test application

[nohwnd]

--test-modules allows you to filter to already built apps and run them via dotnet test. So maybe that is an option for you, if you want to build all and then run just some.

Otherwise there is currently no official way to build a solution and then filter the projects to run only selected projects. Closest is a solution filter file.

[Youssef1313]

Are you using MSTest or NUnit with Microsoft.Testing.Platform? You can try this:

<!-- IsTestProject lets dotnet test command to know this is not a VSTest test project -->
<IsTestProject Condition="'$(SkipDeployedTests)' == 'true'">false</IsTestProject>

<!-- EnableMSTestRunner is the right point for MSTest+MTP to correctly set IsTestingPlatformApplication -->
<!-- This tells dotnet test command to know this is not a Microsoft.Testing.Platform test project -->
<!-- Combined outcome is that this project will be ignored by test command -->
<!-- NOTE:: Use EnableNUnitRunner instead if you use NUnit. -->
<EnableMSTestRunner Condition="'$(SkipDeployedTests)' == 'true'">false</EnableMSTestRunner>

<!-- Also set OutputType to Library based on the same condition, depending on how you set it to Exe -->