MTP v2 with `Microsoft.Testing.Extensions.CodeCoverage`'s `--coverage` produces empty results when running against project built with `ContinuousIntegrationBuild` set to `true`

[julealgon]

Problem

Running MTP v2 with dotnet test --test-submodules **/*.UnitTests.exe --coverage produces a completely empty coverage file when run against executables generated with the ContinuousIntegrationBuild flag set to true

Description

We have a scenario where we execute tests using dotnet --test-modules and pass the .exe files from a dotnet publish operation performed in a separate GitHub actions job.

This was working fine until we tried adding code coverage to the process.

In this scenario, the coverage results are always completely empty, no matter which format we pick.

I've debugged it and found out that the culprit is the ContinuousIntegrationBuild MSBuild property, which we set to true on this particular solution for full SourceLink integration (these are library projects that generate internal NuGet packages): when that setting is set to false, the coverage output is properly populated.

I also noticed that using the dotnet-coverage dotnet tool (instead of using the Microsoft.Testing.Extensions.CodeCoverage package), does work correctly even when ContinuousIntegrationBuild is true.

Why does Microsoft.Testing.Extensions.CodeCoverage not work properly with ContinuousIntegrationBuild? Is there a way to make it work, or is this a bug?

Version Information

We are using latest versions of all tools and packages involved:

• Microsoft.Testing.Extensions.CodeCoverage@18.1.0
• xunit.v3.mtp-v2@3.2.1
• dotnet-coverage@18.1.0

This is a modern .NET 10 solution, and we are using the new MTP runner by setting the global.json value:

  "test": {
    "runner": "Microsoft.Testing.Platform"
  }

The issue seems to occur both on Windows as well as Linux (using **/*.UnitTests.dll instead there).

Workarounds

We will likely migrate the logic to leverage the dotnet-coverage tool in the meantime, but that seems less than ideal considering MTP is brand new and should natively support code coverage with the extension package.

The differences I noticed are that when using dotnet-coverage, I have to explicitly annotate my test projects with the ExcludeFromCodeCoverage attribute otherwise they are included in the report as well, whereas MTP's extension automatically removes the test projects from the results, and that dotnet-coverage will wrap the entire test run and generate a single output file, whereas MTP will generate one file per project. This second one doesn't really affect us as we are using ReportGenerator which works with both.