Testhost.exe does not find '7.0.0-preview.6.22324.4' runtime when running tests

[adityapatwardhan]

Describe the bug

When running xunit tests using dotnet test the building of the product code and test code completes with the correct SDK version - 7.0.0-preview.6.22324.4.

/cc @marcpopMSFT @mmitche

But when test execution begins, an error is reported:

Testhost processId: 27132 exited with exitcode: -2147450730 error: 'It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '7.0.0-preview.6.22324.4' (x64) was not found.

• The following frameworks were found:
  3.1.20 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  3.1.24 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  3.1.26 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  5.0.11 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  5.0.16 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  5.0.17 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  6.0.5 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  6.0.6 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  You can resolve the problem by installing the specified framework and/or SDK.
  The specified framework can be found at:
• https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=7.0.0-preview.6.22324.4&arch=x64&rid=win10-x64

I have attached the debug log from the run: diag2.txt

Workaround: Setting DOTNET_ROOT to point to 'C:\Users\adity\AppData\Local\Microsoft\dotnet' fixes the issue.

This has started happening with .NET 7 Preview 6.

To Reproduce

1. Clone this branch: https://github.com/PowerShell/PowerShell/tree/dotnet_update
2. From PowerShell ->
   Import-Module build.psm1
   Start-PSBootstrap
   cd test\xUnit
   dotnet test

Exceptions (if any)

esthost process exited with error: It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '7.0.0-preview.6.22324.4' (x64) was not found.

• The following frameworks were found:
  3.1.20 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  3.1.24 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  3.1.26 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  5.0.11 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  5.0.16 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  5.0.17 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  6.0.5 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  6.0.6 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  You can resolve the problem by installing the specified framework and/or SDK.
  The specified framework can be found at:
• https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=7.0.0-preview.6.22324.4&arch=x64&rid=win10-x64
  . Please check the diagnostic logs for more information.

Test Run Aborted.

Further technical details

• Include the output of dotnet --info
  .NET SDK:
  Version: 7.0.100-preview.6.22352.1
  Commit: 492644e08e

Runtime Environment:
OS Name: Windows
OS Version: 10.0.22000
OS Platform: Windows
RID: win10-x64
Base Path: C:\Users\adity\AppData\Local\Microsoft\dotnet\sdk\7.0.100-preview.6.22352.1\

Host:
Version: 7.0.0-preview.6.22324.4
Architecture: x64
Commit: d3fa592f6d

.NET SDKs installed:
7.0.100-preview.6.22352.1 [C:\Users\adity\AppData\Local\Microsoft\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.0-preview.6.22330.3 [C:\Users\adity\AppData\Local\Microsoft\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 7.0.0-preview.6.22324.4 [C:\Users\adity\AppData\Local\Microsoft\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 7.0.0-preview.6.22351.3 [C:\Users\adity\AppData\Local\Microsoft\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
D:\PSGit\PowerShell\global.json

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

• The IDE (VS / VS Code/ VS4Mac) you're running on, and its version

[marcpopMSFT]

@pavelhorak @nohwnd to take a look. This is a local install of the SDK running tests and now finding that local install. I'm not sure why this would be expected to work without setting DOTNET_ROOT

[adityapatwardhan]

The expected dotnet.exe is the first thing on the PATH.

[marcpopMSFT]

CC @elinor-fung @vitek-karas with the old MLL behavior, I'd expect it to still search the global location but MLL should be disabled now. Is testhost still using it?

[elinor-fung]

I don't think this is related to multi-level lookup. If I recall correctly, testhost.exe itself is a pretty old apphost - 2.1, I think?

I believe the expectation is that when the test runner (vstest.console.dll) launches testhost, it sets appropriate environment variables (like DOTNET_ROOT) such that that the local install used to run the dotnet test command can be found:
https://github.com/microsoft/vstest/blob/6815879540aa563b0b1f952c07b72d6b98c17201/src/Microsoft.TestPlatform.TestHostProvider/Hosting/DotnetTestHostManager.cs#L469-L475

It looks like that is not happening now.

I suspect this may be from microsoft/vstest#3715
cc @Evangelink

That change sets DOTNET_ROOT_<arch> instead of DOTNET_ROOT when the target version of the test assembly is 6.0+. However, it is testhost that reads those environment variables. Since testhost from pre-6.0, it does not understand or check for DOTNET_ROOT_<arch>.

[iSazonov]

Is this issue only testhost related? A week before I installed latest VS 2022 Community, then .Net 7.0 SDK Preview5 in my profile and now dotnet --list-sdks doesn't enumerate all installed .Net versions based on PATH.

C:\Users\1\Documents\GitHub\iSazonov\runtime [main ≡]> dotnet --list-sdks

7.0.100-preview.5.22307.18 [C:\Users\1\AppData\Local\Microsoft\dotnet\sdk]

If I remove C:\Users\1\AppData\Local\Microsoft\dotnet\sdk from PATH I get:

dotnet --list-sdks
3.1.420 [C:\Program Files\dotnet\sdk]
6.0.400-preview.22301.10 [C:\Program Files\dotnet\sdk] 
7.0.100-preview.4.22252.9 [C:\Program Files\dotnet\sdk]

[pavelhorak]

@MarcoRossignoli can you look at this?

[MarcoRossignoli]

We found the issue and we're working on the fix.

[vitek-karas]

@iSazonov that is because 7.0 disabled multi-level lookup. So if you run dotnet.exe from C:\privatepath\dotnet then it will not try to look into program files or anywhere else. See this design doc for reasoning and exact behavior: https://github.com/dotnet/designs/blob/main/accepted/2022/disable-multi-level-lookup-by-default.md