Unable to run unit tests locally: "Could not load xunit.runner.utility.netcoreapp10"

[carlossanlop]

When I try to run the unit tests from any csproj, I get the error I shared at the end.

Repro steps:

• Synced my runtime enlistment to the latest bits today.
• Built it using build clr+libs+libs.pretest -rc release.
• Tried executing unit tests using dotnet build -c Debug /t:Test .\System.IO.FileSystem.Tests.csproj.

My environment:

• Win10 x64
• Latest dotnet that shows up in dotnet --list-sdks: 6.0.100-preview.5.21277.7 [C:\Program Files\dotnet\sdk]
• Visual Studio version: 16.11.0 Preview 1.0 [31326.366.d16.11

Let me know if you need more info from me.

Microsoft (R) Build Engine version 17.0.0-preview-21274-08+9d419252d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  System.Security.Principal.Windows -> D:\runtime\artifacts\bin\System.Security.Principal.Windows\ref\netstandard2.0-Debug\System.Security.Principal.Windows.dll
  System.Security.AccessControl -> D:\runtime\artifacts\bin\System.Security.AccessControl\ref\netstandard2.0-Debug\System.Security.AccessControl.dll
  Microsoft.Win32.Registry -> D:\runtime\artifacts\bin\Microsoft.Win32.Registry\ref\netstandard2.0-Debug\Microsoft.Win32.Registry.dll
  System.Runtime -> D:\runtime\artifacts\bin\System.Runtime\ref\net6.0-Debug\System.Runtime.dll
  System.Runtime.Extensions -> D:\runtime\artifacts\bin\System.Runtime.Extensions\ref\net6.0-Debug\System.Runtime.Extensions.dll
  System.ComponentModel -> D:\runtime\artifacts\bin\System.ComponentModel\ref\net6.0-Debug\System.ComponentModel.dll
  System.Collections -> D:\runtime\artifacts\bin\System.Collections\ref\net6.0-Debug\System.Collections.dll
  Microsoft.Win32.Primitives -> D:\runtime\artifacts\bin\Microsoft.Win32.Primitives\ref\net6.0-Debug\Microsoft.Win32.Primitives.dll
  System.Security.Principal -> D:\runtime\artifacts\bin\System.Security.Principal\ref\net6.0-Debug\System.Security.Principal.dll
  System.Runtime.InteropServices -> D:\runtime\artifacts\bin\System.Runtime.InteropServices\ref\net6.0-Debug\System.Runtime.InteropServices.dll
  System.Collections.NonGeneric -> D:\runtime\artifacts\bin\System.Collections.NonGeneric\ref\net6.0-Debug\System.Collections.NonGeneric.dll
  System.ObjectModel -> D:\runtime\artifacts\bin\System.ObjectModel\ref\net6.0-Debug\System.ObjectModel.dll
  System.Memory -> D:\runtime\artifacts\bin\System.Memory\ref\net6.0-Debug\System.Memory.dll
  System.ComponentModel.Primitives -> D:\runtime\artifacts\bin\System.ComponentModel.Primitives\ref\net6.0-Debug\System.ComponentModel.Primitives.dll
  System.Net.Primitives -> D:\runtime\artifacts\bin\System.Net.Primitives\ref\net6.0-Debug\System.Net.Primitives.dll
  System.Collections.Specialized -> D:\runtime\artifacts\bin\System.Collections.Specialized\ref\net6.0-Debug\System.Collections.Specialized.dll
  System.Net.WebSockets -> D:\runtime\artifacts\bin\System.Net.WebSockets\ref\net6.0-Debug\System.Net.WebSockets.dll
  System.Security.Claims -> D:\runtime\artifacts\bin\System.Security.Claims\ref\net6.0-Debug\System.Security.Claims.dll
  System.Security.Principal.Windows -> D:\runtime\artifacts\bin\System.Security.Principal.Windows\ref\net6.0-Debug\System.Security.Principal.Windows.dll
  System.Security.AccessControl -> D:\runtime\artifacts\bin\System.Security.AccessControl\ref\net6.0-Debug\System.Security.AccessControl.dll
  TestUtilities -> D:\runtime\artifacts\bin\TestUtilities\net6.0-Debug\TestUtilities.dll
  StreamConformanceTests -> D:\runtime\artifacts\bin\StreamConformanceTests\net6.0-Debug\StreamConformanceTests.dll
  System.IO.FileSystem.Tests -> D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\System.IO.FileSystem.Tests.dll
  ----- start 2021-05-27 17:55:43.09 ===============  To repro directly: =====================================================
  pushd D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\
  "D:\runtime\artifacts\bin\testhost\net6.0-windows-Debug-x64\dotnet.exe" exec --runtimeconfig System.IO.FileSystem.Tests.runtimeconfig.json --depsfile System.IO.FileSystem.Tests.deps.json xunit.console.dll System.IO.FileSystem.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing
  popd
  ===========================================================================================================
  Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.netcoreapp10, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'. The system cannot find the file specified.
  File name: 'xunit.runner.utility.netcoreapp10, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
     at Xunit.ConsoleClient.ConsoleRunner..ctor(Object consoleLock)
     at Xunit.ConsoleClient.Program.Main(String[] args) in /_/src/Microsoft.DotNet.XUnitConsoleRunner/src/Program.cs:line 16
  ----- end 2021-05-27 17:55:48.02 ----- exit code -532462766 ----------------------------------------------------------
D:\runtime\eng\testing\tests.targets(125,5): error : One or more tests failed while running tests from 'System.IO.FileSystem.Tests'. [D:\runtime\src\libraries\System.IO.FileSystem\tests\System.IO.FileSystem.Tests.csproj]

Build FAILED.

@Anipik @ericstj @ViktorHofer

[carlossanlop]

I initially thought this was a problem when attempting to run my unit tests via Visual Studio. After building my enlistment, I:

• Opened Visual Studio. I tried two ways, which were both unsuccessful: opening via the Start menu, or using build.cmd -vs System.IO.FileSystem which has worked sometimes in the past.
• Go to Test Explorer, select "System.IO.FileSystem.Tests (net6.0-windows)", right click it, select "Run"
• When I go to the Output window, the Build output seems to be succesful, but when I switch to the Tests output, this is the error message I see:

[2021-05-27 5:15:16.251 PM] ========== Starting test run ==========
[2021-05-27 5:15:16.267 PM] Tests run settings for D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\System.IO.FileSystem.Tests.dll:
 <RunSettings>
  <RunConfiguration>
    <!-- Timeout in ms, 5 minutes -->
    <TestSessionTimeout>300000</TestSessionTimeout>
    <!-- Directory for test run reports. E.g. trx, coverage etc. -->
    <ResultsDirectory>D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\TestResults\</ResultsDirectory>
    <!-- Working directory for test invocation. Results directory can be relative to this. Used by IDEs. -->
    <SolutionDirectory>.\</SolutionDirectory>
    <!-- Degree of parallelization, spawns n test hosts to run tests. -->
    <MaxCpuCount>0</MaxCpuCount>
    <!-- Configures the architecture of test host. -->
    <TargetPlatform>x64</TargetPlatform>
    <!-- If true, an adapter should disable any test case parallelization. -->
    <DisableParallelization>false</DisableParallelization>
    <!-- If true, an adapter shouldn't create appdomains to run tests. -->
    <DisableAppDomain>false</DisableAppDomain>
    <!-- Filter out failing (wrong framwork, platform, runtime or activeissue) tests -->
    <TestCaseFilter>category!=OuterLoop&amp;category!=failing</TestCaseFilter>
    <DotNetHostPath>D:\runtime\artifacts\bin\testhost\net6.0-windows-Debug-x64\dotnet.exe</DotNetHostPath>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
  <LoggerRunSettings>
    <Loggers>
      <Logger friendlyName="trx" />
      <Logger friendlyName="html" />
      <Logger friendlyName="console">
        <Configuration>
          <Verbosity>Minimal</Verbosity>
        </Configuration>
      </Logger>
    </Loggers>
  </LoggerRunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage" enabled="false">
        <Configuration>
          <Include>[System.IO.FileSystem]*</Include>
          <ExcludeByFile>D:\runtime\src\libraries\Common\src\System\SR.*,D:\runtime\src\libraries\Common\src\System\NotImplemented.cs</ExcludeByFile>
          <IncludeDirectory>D:\runtime\artifacts\bin\testhost\net6.0-windows-Debug-x64\shared\Microsoft.NETCore.App\6.0.0</IncludeDirectory>
          <Format>opencover</Format>
          <SingleHit>false</SingleHit>
          <UseSourceLink>true</UseSourceLink>
          <IncludeTestAssembly>false</IncludeTestAssembly>
        </Configuration>
      </DataCollector>
      <DataCollector friendlyName="blame" enabled="true" />
    </DataCollectors>
  </DataCollectionRunSettings>
  <Python>
    <TestCases />
  </Python>
</RunSettings>.
[2021-05-27 5:15:17.492 PM] Could not find file D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\System.IO.FileSystem.Tests.dll.
[2021-05-27 5:15:17.493 PM] None of the specified source(s) 'D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\System.IO.FileSystem.Tests.dll' is valid. Fix the above errors/warnings and then try again. 
[2021-05-27 5:15:17.695 PM] ========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 3 ms ==========

Tagging subscribers to this area: @Anipik, @safern, @ViktorHofer
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When I try to run the unit tests from any csproj, I get the error I shared at the end.

Repro steps:

• Synced my runtime enlistment to the latest bits today.
• Built it using build clr+libs+libs.pretest -rc release.
• Tried executing unit tests using dotnet build -c Debug /t:Test .\System.IO.FileSystem.Tests.csproj.

My environment:

• Win10 x64
• Latest dotnet that shows up in dotnet --list-sdks: 6.0.100-preview.5.21277.7 [C:\Program Files\dotnet\sdk]
• Visual Studio version: 16.11.0 Preview 1.0 [31326.366.d16.11

Let me know if you need more info from me.

Microsoft (R) Build Engine version 17.0.0-preview-21274-08+9d419252d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  System.Security.Principal.Windows -> D:\runtime\artifacts\bin\System.Security.Principal.Windows\ref\netstandard2.0-Debug\System.Security.Principal.Windows.dll
  System.Security.AccessControl -> D:\runtime\artifacts\bin\System.Security.AccessControl\ref\netstandard2.0-Debug\System.Security.AccessControl.dll
  Microsoft.Win32.Registry -> D:\runtime\artifacts\bin\Microsoft.Win32.Registry\ref\netstandard2.0-Debug\Microsoft.Win32.Registry.dll
  System.Runtime -> D:\runtime\artifacts\bin\System.Runtime\ref\net6.0-Debug\System.Runtime.dll
  System.Runtime.Extensions -> D:\runtime\artifacts\bin\System.Runtime.Extensions\ref\net6.0-Debug\System.Runtime.Extensions.dll
  System.ComponentModel -> D:\runtime\artifacts\bin\System.ComponentModel\ref\net6.0-Debug\System.ComponentModel.dll
  System.Collections -> D:\runtime\artifacts\bin\System.Collections\ref\net6.0-Debug\System.Collections.dll
  Microsoft.Win32.Primitives -> D:\runtime\artifacts\bin\Microsoft.Win32.Primitives\ref\net6.0-Debug\Microsoft.Win32.Primitives.dll
  System.Security.Principal -> D:\runtime\artifacts\bin\System.Security.Principal\ref\net6.0-Debug\System.Security.Principal.dll
  System.Runtime.InteropServices -> D:\runtime\artifacts\bin\System.Runtime.InteropServices\ref\net6.0-Debug\System.Runtime.InteropServices.dll
  System.Collections.NonGeneric -> D:\runtime\artifacts\bin\System.Collections.NonGeneric\ref\net6.0-Debug\System.Collections.NonGeneric.dll
  System.ObjectModel -> D:\runtime\artifacts\bin\System.ObjectModel\ref\net6.0-Debug\System.ObjectModel.dll
  System.Memory -> D:\runtime\artifacts\bin\System.Memory\ref\net6.0-Debug\System.Memory.dll
  System.ComponentModel.Primitives -> D:\runtime\artifacts\bin\System.ComponentModel.Primitives\ref\net6.0-Debug\System.ComponentModel.Primitives.dll
  System.Net.Primitives -> D:\runtime\artifacts\bin\System.Net.Primitives\ref\net6.0-Debug\System.Net.Primitives.dll
  System.Collections.Specialized -> D:\runtime\artifacts\bin\System.Collections.Specialized\ref\net6.0-Debug\System.Collections.Specialized.dll
  System.Net.WebSockets -> D:\runtime\artifacts\bin\System.Net.WebSockets\ref\net6.0-Debug\System.Net.WebSockets.dll
  System.Security.Claims -> D:\runtime\artifacts\bin\System.Security.Claims\ref\net6.0-Debug\System.Security.Claims.dll
  System.Security.Principal.Windows -> D:\runtime\artifacts\bin\System.Security.Principal.Windows\ref\net6.0-Debug\System.Security.Principal.Windows.dll
  System.Security.AccessControl -> D:\runtime\artifacts\bin\System.Security.AccessControl\ref\net6.0-Debug\System.Security.AccessControl.dll
  TestUtilities -> D:\runtime\artifacts\bin\TestUtilities\net6.0-Debug\TestUtilities.dll
  StreamConformanceTests -> D:\runtime\artifacts\bin\StreamConformanceTests\net6.0-Debug\StreamConformanceTests.dll
  System.IO.FileSystem.Tests -> D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\System.IO.FileSystem.Tests.dll
  ----- start 2021-05-27 17:55:43.09 ===============  To repro directly: =====================================================
  pushd D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug\
  "D:\runtime\artifacts\bin\testhost\net6.0-windows-Debug-x64\dotnet.exe" exec --runtimeconfig System.IO.FileSystem.Tests.runtimeconfig.json --depsfile System.IO.FileSystem.Tests.deps.json xunit.console.dll System.IO.FileSystem.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing
  popd
  ===========================================================================================================
  Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.netcoreapp10, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'. The system cannot find the file specified.
  File name: 'xunit.runner.utility.netcoreapp10, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
     at Xunit.ConsoleClient.ConsoleRunner..ctor(Object consoleLock)
     at Xunit.ConsoleClient.Program.Main(String[] args) in /_/src/Microsoft.DotNet.XUnitConsoleRunner/src/Program.cs:line 16
  ----- end 2021-05-27 17:55:48.02 ----- exit code -532462766 ----------------------------------------------------------
D:\runtime\eng\testing\tests.targets(125,5): error : One or more tests failed while running tests from 'System.IO.FileSystem.Tests'. [D:\runtime\src\libraries\System.IO.FileSystem\tests\System.IO.FileSystem.Tests.csproj]

Build FAILED.

@Anipik @ericstj @ViktorHofer

Author:	carlossanlop
Assignees:	-
Labels:	area-Infrastructure-libraries, untriaged
Milestone:	-

[ViktorHofer]

I just validated this scenario and didn't encounter any issues. After building the repo initially via build.cmd clr+libs -rc Release I validated the different supported workloads:

• VS Test Explorer after a build.cmd -vs System.IO.FileSystem
  
• dotnet build /t:Test

    Finished:    System.IO.FileSystem.Tests
  === TEST EXECUTION SUMMARY ===
     System.IO.FileSystem.Tests  Total: 6640, Errors: 0, Failed: 0, Skipped: 17, Time: 12.112s
  ----- end Fri 05/28/2021  9:55:55.38 ----- exit code 0 ----------------------------------------------------------

• dotnet test

Passed!  - Failed:     0, Passed:  6623, Skipped:    17, Total:  6640, Duration: 15 s - System.IO.FileSystem.Tests.dll (net6.0)

My assumption is that the issue you encountered were because of the recent System.IO.FileSystem changes done by @adamsitnik. I faced a similar problem because I didn't clean the repo after that change went in.

[carlossanlop]

@ViktorHofer what additional information can I provide to you for additional help? I already cleaned the enlistment with git clean -fdx and rebuilt it (did this twice), but the issue is still there.

[carlossanlop]

@ViktorHofer I see what you mean. @adamsitnik already merged the PR where System.IO.FileSystem was folded into System.Private.CoreLib.

My enlistment did not yet have those changes, I was a few commits behind.

Do you think the problem is that the SDK that gets installed when building, is the latest nightly, so it contains Adam's changes already? Would that cause a conflict with my enlistment being behind?

[carlossanlop]

Update: No, that wasn't it :(

I synced my enlistment to the latest bits, which includes Adam's changes, ran git clean -fdx, then built everything with the same command, and the unit tests keep failing with the exact same error.

[ericstj]

Do you think the problem is that the SDK that gets installed when building, is the latest nightly, so it contains Adam's changes already? Would that cause a conflict with my enlistment being behind?

That's not how it works. We only update SDK monthly and its currently on Preview3. That's only installed in the local clone .dotnet folder. That SDK used to build is only used to build not run tests. It would be pointless to run tests against something we weren't building (thus the build clr+libs up front).

You have manually installed a build of preview 5 on your machine which you are using when you type "dotnet", but I don't think that's the cause here (I tried it and didn't repro).

One thought is that you have a nuget cache with a bad package in it, just a wild guess.

Can you try doing the following:

rd /q /s %USERPROFILE%\.nuget\packages\microsoft.dotnet.xunitconsolerunner
rd /q /s D:\runtime\artifacts\bin\System.IO.FileSystem.Tests\net6.0-windows-Debug
dotnet build -c Debug /t:Test D:\runtime\src\libraries\System.IO.FileSystem\tests\System.IO.FileSystem.Tests.csproj /bl

If it repros share the binlog. In email if you don't want to expose your environment 📧