Code coverage not 100% on record

[tomap]

Hello,

I'm using dotnet 9
I'm running the tests via dotnet test

• .runsetings file :

<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">   

I use reportgenerator to view the coverage:

And for the record, there is an line matching the declaration of the record that is not covered

<function id="2625" name=".ctor(System.Guid)" namespace="CC.Core.Commands" type_name="GetRicFeedbackCommand" block_coverage="0.00" line_coverage="100.00" blocks_covered="0" blocks_not_covered="0" lines_covered="4" lines_partially_covered="0" lines_not_covered="0">
  <ranges>
    <range source_id="380" start_line="14" end_line="14" start_column="0" end_column="0" covered="yes" />
    <range source_id="380" start_line="15" end_line="15" start_column="0" end_column="0" covered="yes" />
    <range source_id="380" start_line="16" end_line="16" start_column="0" end_column="0" covered="yes" />
    <range source_id="380" start_line="17" end_line="17" start_column="0" end_column="0" covered="yes" />
  </ranges>
</function>
<function id="2626" name="get_WorkspaceKey()" namespace="CC.Core.Commands" type_name="GetRicFeedbackCommand" block_coverage="0.00" line_coverage="100.00" blocks_covered="0" blocks_not_covered="0" lines_covered="1" lines_partially_covered="0" lines_not_covered="0">
  <ranges>
    <range source_id="380" start_line="18" end_line="18" start_column="0" end_column="0" covered="yes" />
  </ranges>
</function>
<function id="2627" name="set_WorkspaceKey(System.Guid)" namespace="CC.Core.Commands" type_name="GetRicFeedbackCommand" block_coverage="0.00" line_coverage="100.00" blocks_covered="0" blocks_not_covered="0" lines_covered="1" lines_partially_covered="0" lines_not_covered="0">
  <ranges>
    <range source_id="380" start_line="18" end_line="18" start_column="0" end_column="0" covered="yes" />
  </ranges>
</function>
<function id="2628" name=".ctor(CC.Core.Commands.GetRicFeedbackCommand)" namespace="CC.Core.Commands" type_name="GetRicFeedbackCommand" block_coverage="0.00" line_coverage="0.00" blocks_covered="0" blocks_not_covered="0" lines_covered="0" lines_partially_covered="0" lines_not_covered="1">
  <ranges>
    <range source_id="380" start_line="12" end_line="12" start_column="0" end_column="0" covered="no" />
  </ranges>
</function>

=> .ctor(CC.Core.Commands.GetRicFeedbackCommand) => covered="no"

I already reported the issue on ReportGenerator project where the conclusion was that the issue came from the cobertura report, not reportgenerator danielpalme/ReportGenerator#749

And it seems coverlet gets it right danielpalme/ReportGenerator#749 (comment)

Does the issue come from Microsoft.CodeCoverage ? Am I posting the issue in the right repo ?
Is it a matter of "interpretation" (I understand that there is some generated "wiring code" generated for records which might not be covered) ?

Thank you for your help

[Youssef1313]

I think microsoft/codecoverage should be the right repo. Moving there.

[fhnaseer]

A copy constructor is generated in case of record and for that coverage is zero.

This can be skipped if you set SkipAutoProperties to true in runsettings file. However, this will be available in 18.0 version of code coverage which is not released yet. As a workaround, you can try to exclude this function from the coverage report using runsettings file. https://github.com/microsoft/codecoverage/blob/main/docs/configuration.md

[tomap]

Hello,

Thank you for the advice, I've added the property, which will work with 18.0

In the meantime, I've added

<Attributes>
  <Exclude>
    <Attribute>^System\.Runtime\.CompilerServices\.CompilerGeneratedAttribute$</Attribute>
  </Exclude>
</Attributes>

It also works

[fhnaseer]

Please keep this in mind that System\.Runtime\.CompilerServices\.CompilerGeneratedAttribute will also exclude those functions for which you might want to have coverage e.g. lambdas.

[Lorilatschki]

@fhnaseer we having the same issue and try to get rid of this behavior. We tried to use the suggested setting SkipAutoProperties but with no effect. We are using version 18.0.6 of Microsoft.Testing.Extensions.CodeCoverage.

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
    <CodeCoverage>
        <SkipAutoProperties>True</SkipAutoProperties>
    </CodeCoverage>
</Configuration>

The above didn't worked, but this does:

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
    <CodeCoverage>
        <Attributes>
            <Exclude>
                <Attribute>^System\.Runtime\.CompilerServices\.CompilerGeneratedAttribute$</Attribute>
            </Exclude>
        </Attributes>
    </CodeCoverage>
</Configuration>

Is there a way to change the default setting so that automatically generated properties/methods are ignored, as I cannot imagine that the majority would want such a metric to be part of the coverage data?

We do have many small repos in our project and switching it off by providing such a settings file for each and every repo is kind of to much effort. We have a central managed CI/CD pipeline, therefor a CLI parameter or a changed default value would help a lot.

[fhnaseer]

I created a new MSTest 3.11.1 project with MS.Testing.Ext.CC 18.0.4 and is working fine.

SkipAutoProperties = true

SkipAutoProperties = false

Changing default to true will reduce number of lines in the coverage report which can affect users in different ways.

^System\.Runtime\.CompilerServices\.CompilerGeneratedAttribute$ can also skip some other methods for which you want to calculate coverage for. In the CI/CD you can have one runsettings file for your whole repo and can use it to have consistent results.

[Lorilatschki]

@fhnaseer in your example you are using the <RunSettings> structure, not the simple subset of such a .runsettings file. Can you try use exactly this one?

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
    <CodeCoverage>
        <SkipAutoProperties>True</SkipAutoProperties>
    </CodeCoverage>
</Configuration>

[fhnaseer]

It should work same for both cases (full runsettings or only configuration).

[Lorilatschki]

This doesn't work on our end. When running our tests with that, still the copy ctor is part of the coverage file. Maybe this is related to the used versions of dotnet and MTP, isn't it?
We are using the following versions:

• xunit.v3 with version v3.2.1
• dotnet 8
• Microsoft.NET.Test.Sdk with version 18.0.1
• MTP with version 1.9.0

Configuration File coverage.config:

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
    <CodeCoverage>
        <SkipAutoProperties>True</SkipAutoProperties>
    </CodeCoverage>
</Configuration>

Test execution output:

xUnit.net v3 Microsoft.Testing.Platform v1 Runner v3.2.1+a9cfb80929 (64-bit .NET 8.0.22)


  In Bearbeitung Dateiartefakte erstellt:
    - test-results\coverage.cobertura.xml

Testlaufzusammenfassung: Bestanden! - test\MyTest.dll (net8.0|x64)
  total: 180
  fehlerhaft: 0
  succeeded: 180
  übersprungen: 0
  duration: 24s 558ms

Test exeuction CLI:
dotnet run --project test/myTest.csproj -c Release --no-restore --coverage --coverage-output-format cobertura --coverage-output coverage.cobertura.xml --results-directory ./test-result --no-progress --coverage-settings .\coverage.config

The final coverage.cobertura.xml still has this line:

<class line-rate="0.8888888888888888" branch-rate="1" complexity="2" name="CliArgument" filename=".\CliArgument.cs">
  <methods>
	<method line-rate="0" branch-rate="1" complexity="1" name=".ctor" signature="(CliArgument)">
	  <lines>
		<line number="10" hits="0" branch="False" />
	  </lines>
	</method>
  </methods>
</class>

[Lorilatschki]

@fhnaseer regarding

In the CI/CD you can have one runsettings file for your whole repo and can use it to have consistent results.

Nope, this is not that easy. As I said, we have 40+ repos, most of them don't have a coverage.config file defined, because there was currently no need for. Some already define such a coverage setting file, because they needed to override some ModulePath which is unique for every repo. Therefore, we cannot easily use one central coverage setting file and we also don't want to introduce such a coverage setting file for each and every repo, just to get auto generated stuff hidden in the coverage result file.
In our opinion, auto generated stuff should be opt out rather than opt in.

[fhnaseer]

Its working fine for xunit as well.

<PropertyGroup>
	<TargetFramework>net8.0</TargetFramework>
	<ImplicitUsings>enable</ImplicitUsings>
	<Nullable>enable</Nullable>

	<OutputType>Exe</OutputType>
	<UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
	<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
	<TestingPlatformShowTestsFailure>true</TestingPlatformShowTestsFailure>
</PropertyGroup>
<ItemGroup>
	<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
	
	<PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" Version="18.0.6" />
	<PackageReference Include="xunit.v3" Version="3.2.1" />
	<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5">
	  <PrivateAssets>all</PrivateAssets>
	  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
	</PackageReference>
</ItemGroup>

<ItemGroup>
	<Using Include="Xunit" />
</ItemGroup>

<ItemGroup>
  <None Update="coverage.config">
    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
  </None>
</ItemGroup>