Coverlet in-process collector is not loaded for version > 1.0.0 #2221

MarcoRossignoli · 2019-10-10T15:19:57Z

Fixes for #2205

cc: @tonerdo

msftclas · 2019-10-10T15:20:11Z

All CLA requirements met.

mayankbansal018 · 2019-10-21T10:58:06Z

@vagisha-nidhi can you please look

vagisha-nidhi · 2019-10-21T11:46:16Z

@MarcoRossignoli What is the status of this PR?

MarcoRossignoli · 2019-10-21T11:49:01Z

@vagisha-nidhi I would like to test it on my test project...but I don't know the correct steps, can you link me some guide or describe how I can use my local build of test host?

MarcoRossignoli · 2019-10-23T07:29:44Z

ping @mayankbansal018 @vagisha-nidhi #2221 (comment)

vagisha-nidhi · 2019-10-23T08:38:01Z

@MarcoRossignoli You can gereate the local binaries by using build.cmd on the root folder. This will generate artifacts/Debug/net451/win7-x64 which contain all built binaries. You can just test your dll with vstest.console.exe abc.dll from this folder

MarcoRossignoli · 2019-10-23T08:49:31Z

You can gereate the local binaries by using build.cmd on the root folder

@vagisha-nidhi if I want to update and re-test should I rebuild all or I can rebuild only specific project in visual studio?

MarcoRossignoli · 2019-10-23T16:00:02Z

@vagisha-nidhi I cannot load collectors

C:\git\vstest\artifacts\Debug\net451\win7-x64 (loadcoverlet -> origin)
λ vstest.console.exe "C:\git\coverletissue\collectorTest\XUnitTestProject1\bin\Debug\netcoreapp3.0\XUnitTestProject1.dll" --collect:"XPlat Code Coverage"
Microsoft (R) Test Execution Command Line Tool Version 16.4.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Data collection : Unable to find a datacollector with friendly name 'XPlat Code Coverage'.

Data collection : Could not find data collector 'XPlat Code Coverage'

[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.0 (64-bit .NET Core 3.0.0)

[xUnit.net 00:00:00.65]   Discovering: XUnitTestProject1

[xUnit.net 00:00:00.71]   Discovered:  XUnitTestProject1

[xUnit.net 00:00:00.71]   Starting:    XUnitTestProject1

[xUnit.net 00:00:00.86]   Finished:    XUnitTestProject1

  V XUnitTestProject1.UnitTest1.Test1 [4ms]


Test Run Successful.
Total tests: 1
     Passed: 1
 Total time: 1,8512 Seconds

C:\git\vstest\artifacts\Debug\net451\win7-x64 (loadcoverlet -> origin)

vagisha-nidhi · 2019-10-23T17:10:18Z

You can gereate the local binaries by using build.cmd on the root folder

@vagisha-nidhi if I want to update and re-test should I rebuild all or I can rebuild only specific project in visual studio?

Just build the particular project where your changes are and replace the net451 binaries in the artifacts folder.

vagisha-nidhi · 2019-10-23T17:22:19Z

@vagisha-nidhi I cannot load collectors

C:\git\vstest\artifacts\Debug\net451\win7-x64 (loadcoverlet -> origin)
λ vstest.console.exe "C:\git\coverletissue\collectorTest\XUnitTestProject1\bin\Debug\netcoreapp3.0\XUnitTestProject1.dll" --collect:"XPlat Code Coverage"
Microsoft (R) Test Execution Command Line Tool Version 16.4.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Data collection : Unable to find a datacollector with friendly name 'XPlat Code Coverage'.

Data collection : Could not find data collector 'XPlat Code Coverage'

[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.0 (64-bit .NET Core 3.0.0)

[xUnit.net 00:00:00.65]   Discovering: XUnitTestProject1

[xUnit.net 00:00:00.71]   Discovered:  XUnitTestProject1

[xUnit.net 00:00:00.71]   Starting:    XUnitTestProject1

[xUnit.net 00:00:00.86]   Finished:    XUnitTestProject1

  V XUnitTestProject1.UnitTest1.Test1 [4ms]


Test Run Successful.
Total tests: 1
     Passed: 1
 Total time: 1,8512 Seconds

C:\git\vstest\artifacts\Debug\net451\win7-x64 (loadcoverlet -> origin)

Oh I got it. The coverlet.collector targets file sets the test adapter path to its nuget location. This will happen only when dotnet test is being run. So you won't be able to load the collector. As far as I remember coverlet.collector with vstest.console.exe is broken due to some other reason as well.
For validating for dotnet test, what you can do is (since these inproc collector changes are inside test host only), replace these dlls in microsoft.testplatform.testhost nuget package. Make sure the version you are referring to is the version you have replaced the dlls in and the binaries should be netcore ones.
The location will be something like C:\Users\vanidhi\.nuget\packages\microsoft.testplatform.testhost\16.3.0\lib\netcoreapp2.1

MarcoRossignoli · 2019-10-24T12:38:57Z

@vagisha-nidhi did a test...I used my build package and updated custom nuget.config and overwritten Microsoft.TestPlatform.CrossPlatEngine to attach and all seem work as expected.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="Local" value="C:\git\coverlet\bin\Debug\Packages" />
    <add key="vstest" value="C:\git\vstest\artifacts\Debug\packages" />
    <add key="nuget" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>

...
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0-dev" />
    <PackageReference Include="xunit" Version="2.4.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" />
    <PackageReference Include="coverlet.collector" Version="1.1.0" />
  </ItemGroup>
...

You can do a review.
One question, is there a way to dogfood vstest packages after eventual merge?

src/Microsoft.TestPlatform.Common/Resources/Resources.Designer.cs

MarcoRossignoli · 2019-10-24T12:43:13Z

.../Microsoft.TestPlatform.CrossPlatEngine.UnitTests/DataCollection/InProcDataCollectorTests.cs

@@ -79,6 +79,27 @@ public void InProcDataCollectorShouldInitializeIfAssemblyContainsAnyInProcDataCo
            Assert.AreEqual(this.inProcDataCollector.AssemblyQualifiedName, typeInfo.AssemblyQualifiedName);
        }

+        [TestMethod]
+        public void InProcDataCollectorLoadCoverlet()


this test verify that also if aqn shows version 1.0.0.0 we correctly load a version 9.9.9.9 of lib

MarcoRossignoli · 2019-10-24T12:44:04Z

test/TestAssets/coverlet.collector/CoverletInProcDataCollector.cs

@@ -0,0 +1,40 @@
+using System.Reflection;


Added custom test assets, fake coverlet.collector.dll lib

MarcoRossignoli · 2019-10-28T16:26:58Z

ping @vagisha-nidhi for review...maybe we can fix this before 3.1 release.

ViktorHofer · 2019-10-31T15:56:08Z

@vagisha-nidhi @singhsarab can somebody please take a look? Marco asked me to ping you as this impacts coverlet. thanks.

singhsarab · 2019-10-31T16:38:28Z

@vagisha-nidhi is OOF this week.
@mayankbansal018 you might want to take a look as well.

singhsarab · 2019-10-31T16:55:19Z

test/TestAssets/coverlet.collector/CoverletInProcDataCollector.cs

+using Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollector.InProcDataCollector;
+using Microsoft.VisualStudio.TestPlatform.ObjectModel.InProcDataCollector;
+
+[assembly: AssemblyKeyFile("key.snk")]


What is this key file for?

Mainly 2 reason

Without signed assembly I cannot compile I get an error that force me to have a strong name asm.

After this merge we have on backlog this Sign collector and update visibility coverlet-coverage/coverlet#566 so I'll update this key here and will amend test and alg to check also "public key token".

singhsarab · 2019-10-31T16:57:32Z

src/Microsoft.TestPlatform.CrossPlatEngine/DataCollection/InProcDataCollector.cs

+            {
+                // If we're loading coverlet collector we skip to check the version of assembly
+                // to allow upgrade throught nuget package
+                filterPredicate = (x) => x.FullName.Equals(Constants.CoverletDataCollectorTypeName) && interfaceTypeInfo.IsAssignableFrom(x.GetTypeInfo());


@mayankbansal018 Should we make this default ?

I think no, If I'm not mistaken that aqn is read from possible runsetting file so a user should have that assembly in local folder. The issue with coverlet is that we(you) inject hardcoded aqn if user provide --collect:"XPlat Code Coverage".
It works well for user provided collectors because there is always a runsettings file provided, but with coverlet don't, so we need a way to make it works for every version without runsettings file.
So my idea is to check only "trusted" type/lib name only for coverlet.
BTW I could miss something in the big picture.

MarcoRossignoli · 2019-11-06T08:22:14Z

ping @vagisha-nidhi @singhsarab @mayankbansal018

mayankbansal018 · 2019-11-06T08:38:44Z

@MarcoRossignoli Vagisha is OOF this week, she should be able to pick this up coming Monday, we apologize for the delay.

MarcoRossignoli · 2019-11-06T12:10:01Z

@singhsarab I resolved the conflicts there was an error on string format.

vagisha-nidhi

MarcoRossignoli · 2019-11-13T07:55:43Z

@vagisha-nidhi is there a way to dogfood vstest packages?

vagisha-nidhi · 2019-11-14T06:00:13Z

@MarcoRossignoli Here is the myget link where the recent build gets uploaded https://dotnet.myget.org/feed/vstest/package/nuget/Microsoft.NET.Test.Sdk/16.5.0-preview-20191113-01
Also, I'll be releasing it today so you can use the release build as well.

MarcoRossignoli · 2019-11-14T07:07:44Z

Thanks!

MarcoRossignoli · 2019-11-14T08:12:37Z

Also, I'll be releasing it today so you can use the release build as well.

Great!

MarcoRossignoli added 3 commits October 10, 2019 16:32

updates

1e3b893

updates

ace24d7

updates

c4a4f72

singhsarab requested a review from vagisha-nidhi October 10, 2019 15:29

updates

1fdba7f

updates

99c243b

MarcoRossignoli changed the title ~~[WIP]Coverlet in-process collector is not loaded for version > 1.0.0~~ Coverlet in-process collector is not loaded for version > 1.0.0 Oct 24, 2019

MarcoRossignoli commented Oct 24, 2019

View reviewed changes

src/Microsoft.TestPlatform.Common/Resources/Resources.Designer.cs Show resolved Hide resolved

MarcoRossignoli commented Oct 24, 2019

View reviewed changes

MarcoRossignoli mentioned this pull request Oct 30, 2019

how to use coverlet as project in a solution coverlet-coverage/coverlet#600

Closed

singhsarab requested a review from mayankbansal018 October 31, 2019 16:37

singhsarab reviewed Oct 31, 2019

View reviewed changes

MarcoRossignoli added 2 commits October 31, 2019 19:21

address feedback

a165d0f

Improve test

d72bbeb

singhsarab approved these changes Nov 6, 2019

View reviewed changes

merge master

6b86187

vagisha-nidhi approved these changes Nov 13, 2019

View reviewed changes

singhsarab merged commit ca24ad0 into microsoft:master Nov 13, 2019

MarcoRossignoli deleted the loadcoverlet branch November 13, 2019 08:41

This was referenced Nov 16, 2019

Coverlet in-process collector is not loaded for version > 1.0.0 coverlet-coverage/coverlet#576

Closed

Update know issue and docs on in proc collectors coverlet-coverage/coverlet#620

Closed

MarcoRossignoli mentioned this pull request Dec 19, 2019

[BUG] vstest.console.exe --collect:"XPlat Code Coverage" throw System.FormatException #2278

Closed

This was referenced Jan 25, 2020

Fix coverage SixLabors/ImageSharp#1094

Merged

0% coverage reported on netfx target coverlet-coverage/coverlet#705

Closed

MarcoRossignoli mentioned this pull request Mar 29, 2020

Issue 736 fix attempt experiment coverlet-coverage/coverlet#754

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Coverlet in-process collector is not loaded for version > 1.0.0 #2221

Coverlet in-process collector is not loaded for version > 1.0.0 #2221

MarcoRossignoli commented Oct 10, 2019 •

edited

Loading

msftclas commented Oct 10, 2019 •

edited

Loading

mayankbansal018 commented Oct 21, 2019

vagisha-nidhi commented Oct 21, 2019

MarcoRossignoli commented Oct 21, 2019

MarcoRossignoli commented Oct 23, 2019

vagisha-nidhi commented Oct 23, 2019

MarcoRossignoli commented Oct 23, 2019

MarcoRossignoli commented Oct 23, 2019 •

edited by vagisha-nidhi

Loading

vagisha-nidhi commented Oct 23, 2019

vagisha-nidhi commented Oct 23, 2019

MarcoRossignoli commented Oct 24, 2019 •

edited

Loading

MarcoRossignoli Oct 24, 2019

MarcoRossignoli Oct 24, 2019

MarcoRossignoli commented Oct 28, 2019 •

edited

Loading

ViktorHofer commented Oct 31, 2019

singhsarab commented Oct 31, 2019

singhsarab Oct 31, 2019

MarcoRossignoli Oct 31, 2019 •

edited

Loading

singhsarab Oct 31, 2019

MarcoRossignoli Oct 31, 2019

MarcoRossignoli commented Nov 6, 2019

mayankbansal018 commented Nov 6, 2019

MarcoRossignoli commented Nov 6, 2019

vagisha-nidhi left a comment

MarcoRossignoli commented Nov 13, 2019

vagisha-nidhi commented Nov 14, 2019

MarcoRossignoli commented Nov 14, 2019

MarcoRossignoli commented Nov 14, 2019

Coverlet in-process collector is not loaded for version > 1.0.0 #2221

Coverlet in-process collector is not loaded for version > 1.0.0 #2221

Conversation

MarcoRossignoli commented Oct 10, 2019 • edited Loading

msftclas commented Oct 10, 2019 • edited Loading

mayankbansal018 commented Oct 21, 2019

vagisha-nidhi commented Oct 21, 2019

MarcoRossignoli commented Oct 21, 2019

MarcoRossignoli commented Oct 23, 2019

vagisha-nidhi commented Oct 23, 2019

MarcoRossignoli commented Oct 23, 2019

MarcoRossignoli commented Oct 23, 2019 • edited by vagisha-nidhi Loading

vagisha-nidhi commented Oct 23, 2019

vagisha-nidhi commented Oct 23, 2019

MarcoRossignoli commented Oct 24, 2019 • edited Loading

MarcoRossignoli Oct 24, 2019

Choose a reason for hiding this comment

MarcoRossignoli Oct 24, 2019

Choose a reason for hiding this comment

MarcoRossignoli commented Oct 28, 2019 • edited Loading

ViktorHofer commented Oct 31, 2019

singhsarab commented Oct 31, 2019

singhsarab Oct 31, 2019

Choose a reason for hiding this comment

MarcoRossignoli Oct 31, 2019 • edited Loading

Choose a reason for hiding this comment

singhsarab Oct 31, 2019

Choose a reason for hiding this comment

MarcoRossignoli Oct 31, 2019

Choose a reason for hiding this comment

MarcoRossignoli commented Nov 6, 2019

mayankbansal018 commented Nov 6, 2019

MarcoRossignoli commented Nov 6, 2019

vagisha-nidhi left a comment

Choose a reason for hiding this comment

MarcoRossignoli commented Nov 13, 2019

vagisha-nidhi commented Nov 14, 2019

MarcoRossignoli commented Nov 14, 2019

MarcoRossignoli commented Nov 14, 2019

MarcoRossignoli commented Oct 10, 2019 •

edited

Loading

msftclas commented Oct 10, 2019 •

edited

Loading

MarcoRossignoli commented Oct 23, 2019 •

edited by vagisha-nidhi

Loading

MarcoRossignoli commented Oct 24, 2019 •

edited

Loading

MarcoRossignoli commented Oct 28, 2019 •

edited

Loading

MarcoRossignoli Oct 31, 2019 •

edited

Loading