Restore.cmd does not produce the same outputs as NuGet Restore in Visual Studio

[sharwell]

• This issue is blocking
• This issue is causing unreasonable pain

This issue causes significant performance degradation inside Visual Studio by forcing the entire workspace to be invalidated following a command line build. See dotnet/roslyn#44523 and dotnet/roslyn#44034.

During #5172 I fixed all the known discrepancies. Since that time, several new discrepancies have been introduced. It does not appear possible to maintain state here without either redirecting Visual Studio to use a different restore script or somehow adding tests for this.

[tmat]

Can you check whether dotnet restore Roslyn.sln produces same outputs as restore in VS? If it does not then this issue should be moved to NuGet. Otherwise it's in fact an Arcade issue.

[sharwell]

dotnet restore .\Roslyn.sln produces a bunch of errors like this:

C:\dev\roslyn\eng\targets\GeneratePkgDef.targets(65,99): error MSB4022: The result "" of evaluating the value "$(ArcadeVisualStudioBuildTasksAssembly)" of the "AssemblyFile" attribute in element <UsingTask> is not valid. [C:\dev\roslyn\src\VisualStudio\Setup\Roslyn.VisualStudio.Setup.csproj]

It does not produce any output.

[sharwell]

After modifying VisualStudio.targets to conditionally import GeneratePkgDef.targets only when ArcadeVisualStudioBuildTasksAssembly is defined, dotnet restore .\Roslyn.sln produces a third output which does not match VS or arcade.

[tmat]

Oh, doh. We have VSIX projects, so we can't use dotnet restore. We should condition including GeneratePkgDef.targets on building with desktop msbuild ('$(MSBuildRuntimeType)' != 'Core').

This should work: msbuild /t:restore Roslyn.sln

[tmat]

That said, dotnet restore should work on Compilers.sln and produce same outputs as VS. If not then it's a NuGet bug.

[sharwell]

MSBuild and Arcade do not produce the same outputs. Not counting the items already fixed by #5172 (which never propagated to Roslyn), the following discrepancies are noted, with arcade output as the baseline:

1. runtimeIdentifierGraphPath

   -"runtimeIdentifierGraphPath": "C:\\dev\\roslyn\\.dotnet\\sdk\\3.1.200\\RuntimeIdentifierGraph.json"
   +"runtimeIdentifierGraphPath": "C:\\Program Files\\dotnet\\sdk\\5.0.100-preview.2.20120.3\\RuntimeIdentifierGraph.json"

2. downloadDependencies (exact change varies by project)

   +"downloadDependencies": [
   +  {
   +    "name": "Microsoft.AspNetCore.App.Ref",
   +    "version": "[3.1.0, 3.1.0]"
   +  }
   +],

Since the NuGet up-to-date check is hash-based, all differences under a binary comparison must be eliminated before the Arcade and VS implementations can be enabled at the same time.

[sharwell]

MSBuild and VS produce almost the same output, but there differences in two projects:

1. Microsoft.CodeAnalysis.EditorFeatures.Text has a difference in project.nuget.cache due to a reordering of one of the elements of expectedPackageFiles

2. VS.ExternalAPIs.Roslyn.Package has a difference in project.assets.json due to the handling of two of the expression evaluator projects. It ends up with a couple of these along with the corresponding framework and dependencies differences:

    "Microsoft.CodeAnalysis.ExpressionEvaluator.ResultProvider/3.7.0-dev": {
      "type": "project",
   -  "path": "../../ExpressionEvaluator/Core/Source/ResultProvider/Portable/Microsoft.CodeAnalysis.ResultProvider.csproj",
   -  "msbuildProject": "../../ExpressionEvaluator/Core/Source/ResultProvider/Portable/Microsoft.CodeAnalysis.ResultProvider.csproj"
   +  "path": "../../ExpressionEvaluator/Core/Source/ResultProvider/NetFX20/ResultProvider.NetFX20.csproj",
   +  "msbuildProject": "../../ExpressionEvaluator/Core/Source/ResultProvider/NetFX20/ResultProvider.NetFX20.csproj"
    },

[tmat]

@nkolev92 Seems like there are some bugs in NuGet.

[nkolev92]

Looking at the issues @sharwell has identified.

1. Microsoft.CodeAnalysis.EditorFeatures.Text has a difference in project.nuget.cache due to a reordering of one of the elements of expectedPackageFiles

definitely looks like something NuGet should fix.

For

MSBuild and Arcade do not produce the same outputs. Not counting the items already fixed by #5172 (which never propagated to Roslyn), the following discrepancies are noted, with arcade output as the baseline:

It'd be cool if dotnet/sdk#8254 is implemented. That'd allow VS to use whatever arcade downloads.

Are you all aware of any related efforts to that?

I'd have to take a deeper look at the scenario with the evaluator projects.

[tmat]

Re

MSBuild and Arcade do not produce the same outputs

I assume you mean msbuild Roslyn.sln and build.cmd, correct? Odd, build.ps1 uses msbuild by default so that should match. msbuild is found like so: https://github.com/dotnet/roslyn/blob/master/eng/common/tools.ps1#L247. Not sure why would it find a different SDK than VS though.

[sharwell]

I assume you mean msbuild Roslyn.sln and build.cmd, correct?

Yes. I'm referring to msbuild /t:Restore as compared to Restore.cmd.

[tmat]

@nkolev92 Would you mind filing issues that you see here that NuGet team would be responsible for fixing? We need to be able to identify all components that contribute to the problem and track progress in fixing them.

[tmat]

Re the different .NET SDK used during command line build vs VS: I think setting rollForward to latestPatch would address that, but it's not ideal because then we are forcing everyone to install exact version (modulo patches) of .NET SDK that's specified in global.json.

[garath]

Triage: Removing blocking checkbox. @nkolev92, is there a tracking issue?