Restore fails if netstandard1.x package reference is present

[clairernovotny]

SDK version: 3.0.100-preview4-010963

repro:

• clone https://github.com/RssBandit/RssBandit
• cd source/RssBandit
• dotnet restore

  Persisting no-op dg to C:\dev\RssBandit\source\ChildProjects\IEControl\obj\IEControl.csproj.nuget.dgspec.json
  Persisting no-op dg to C:\dev\RssBandit\source\RssBandit.AppServices\obj\RssBandit.AppServices.csproj.nuget.dgspec.json
  Persisting no-op dg to C:\dev\RssBandit\source\NewsComponents\obj\NewsComponents.csproj.nuget.dgspec.json
  Persisting no-op dg to C:\dev\RssBandit\source\RssBandit\obj\RssBandit.csproj.nuget.dgspec.json
  Restore completed in 22.85 ms for C:\dev\RssBandit\source\RssBandit.AppServices\RssBandit.AppServices.csproj.
  Restore completed in 22.83 ms for C:\dev\RssBandit\source\ChildProjects\IEControl\IEControl.csproj.
  Restore completed in 26.1 ms for C:\dev\RssBandit\source\NewsComponents\NewsComponents.csproj.
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Console 4.0.0 -> runtime.win.System.Console 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.Extensions from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.IO.FileSystem 4.0.1 -> runtime.win.System.IO.FileSystem 4.3.0 -> System.Runtime.Extensions (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.Extensions (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.IO.FileSystem 4.0.1 -> runtime.win.System.IO.FileSystem 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.IO.FileSystem from 4.3.0 to 4.0.1. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.IO.FileSystem (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.IO.FileSystem (>= 4.0.1)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Net.NameResolution from 4.3.0 to 4.0.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.Net.NameResolution (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.NameResolution (>= 4.0.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Net.Primitives from 4.3.0 to 4.0.11. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.Net.Primitives (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> System.Net.Primitives (>= 4.0.11)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.Extensions from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.Runtime.Extensions (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.Extensions (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Diagnostics.Process 4.1.0 -> Microsoft.Win32.Primitives 4.0.1 -> runtime.win.Microsoft.Win32.Primitives 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.IO.FileSystem.Watcher 4.0.0 -> Microsoft.Win32.Primitives 4.0.1 -> runtime.win.Microsoft.Win32.Primitives 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.Extensions from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.NameResolution 4.0.0 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> System.Runtime.Extensions (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.Extensions (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.NameResolution 4.0.0 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.Extensions from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Requests 4.0.11 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> System.Runtime.Extensions (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.Extensions (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Requests 4.0.11 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.Extensions from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> System.Runtime.Extensions (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.Extensions (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.Sockets 4.1.0 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Net.NameResolution 4.0.0 -> System.Security.Principal.Windows 4.0.0 -> Microsoft.Win32.Primitives 4.0.1 -> runtime.win.Microsoft.Win32.Primitives 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) 
C:\dev\RssBandit\source\RssBandit\RssBandit.csproj : error NU1605:  RssBandit -> NewsComponents -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
  Restore failed in 467.8 ms for C:\dev\RssBandit\source\RssBandit\RssBandit.csproj.

EDIT by @dsplaisted: This is the workaround for this bug:

<PackageReference Include="Microsoft.NETCore.Targets" Version="2.1.0" ExcludeAssets="all" PrivateAssets="all" />

[nguerrera]

Referencing current HEAD in case it moves between now and repro investigation: RssBandit/RssBandit@628bd5c

[nguerrera]

I was confused on the email thread, this isn't conflict resolution as it's a downgrade during restore.

[dsplaisted]

This should be fixed when we do https://github.com/dotnet/cli/issues/10105. Previously I thought that trimming previous NuGet packages would merely be a perf optimization and conflict resolution would handle selecting the right assets. However this demonstrates a case where there is a functional failure if we don't trim previous packages an old bug can resurface with .NET Core 1.x package references when a RID is specified.

@onovotny I believe adding the following PackageReference to RssBandit.csproj should work around the issue:

<PackageReference Include="Microsoft.NETCore.App" Version="2.2.0" ExcludeAssets="all" PrivateAssets="all" />

[clairernovotny]

Do I want version 2.2.0 or 3.0.0?

[dsplaisted]

2.2.0. We will likely not have this package on NuGet.org for 3.0.0.

[jzabroski]

@dsplaisted The obvious, simpler, solution to this is to implement CleanFull, which we discussed in a separate topic last week as a meta-step I used when upgrading from .NET Core 2.1 to 2.2: #3046 (comment)

One logical reason could be that I define a "cleanfull" target in all my .NET Core projects that does the following:

• Runs MSBuild Clean target
• Runs custom task to remove all nuget packages
• Runs custom task to remove MSTest results directory
• Runs custom task to remove bin and obj directories

Trimming the NuGet object graph is the "clever" solution, but probably also too clever by half and adds complexity to an already complex sub-system.

I've created a ticket for this suggestion: #3067

[jzabroski]

Nevermind. I see that the issue here is that the packagereference is in the csproj. I misunderstood @nguerrera link to the git commit hash - it's the commit that fixes the issue as well as what caused the issue in the first place. I thought @onovotny was saying that he upgraded to 2.0.4 and still got errors due to cached packages still existing in the packages directory.

[dsplaisted]

A simple repro for this is simply a .NET Core console app with a RuntimeIdentifier and a reference to the log4net package:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <RuntimeIdentifier>win-x86</RuntimeIdentifier>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="log4net" Version="2.0.8" />
  </ItemGroup>

</Project>

The problem is that the log4net package depends on System.Runtime.Serialization.Formatters version 4.3.0, but on the 4.0.x or 4.1.x versions of a lot of other contracts. With a RID specified, RID-specific dependencies get injected which cause package downgrades such as the following:

error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version.
error NU1605: consoletest -> log4net 2.0.8 -> System.Console 4.0.0 -> runtime.win.System.Console 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0)
error NU1605: consoletest -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)

To fix this, we may need something like NuGet/Home#7344. We thought an "empty" Microsoft.NETCore.App package could help with this, but because it doesn't have a runtime.json with RID-specific references to the 2.x equivalents of "runtime packs", it didn't fix the issue in the same way that the explicit reference to the 2.2.0 package suggested as a workaround here did.

[jzabroski]

Warning: The value of this comment might be very low.

I'd be careful about over-engineering this. Looking at .NET Core 3.0, I'm already starting to feel like library maintainers will be screwed once more. A few weeks ago I saw Brice Lambson tweeting about how awesome .netstandard2.1 was now that he could write once target everywhere. But since the direction is for .netstandard to be about "API surface area", many of us are upset at the complexity this will introduce in our projects: We see it as going back to the dark ages once more. For example, I can't "just" take the Span type in a new project as a dependency.

To be honest, I don't know what the right solution is and I don't envy the intrepid soul who is tasked with solving the problem, since some of it is idiosyncratic (how binding redirection only works on strong named assemblies and how strong name assemblies came about prior to the advent of the web semver2.0.0 standard and even Java's OSGI standards).

For me, I am filing this case away as interesting and something to noodle on.

[dsplaisted]

@jzabroski The complexity in this issue is coming from trying to make things simpler and better ("less over-engineered") in .NET Core 3.0, while still supporting packages that were created back in .NET Core 1.x when .NET Core was represented by a complex graph of NuGet packages.

[jzabroski]

I may be wrong, but I think the "trivially hard" case is the one I run into regularly:

• You have a TaskRunner tool that targets version 3.1.3 of your project
• Your TaskRunner tool has an --assemblies parameter that loads assemblies that implements an ITask interface exposed by the TaskRunner
• Those assemblies have to match exactly on version number or it won't be able to find the types
  • It seems like there may be a few workarounds but none of them are good. BenchmarkDotNet uses a clever one called DirtyAssemblyResolveHelper for a reason.

In other words, while you may solve it for the framework (which is clever), how will you solve it for the more common scenarios? If you think that's not a big problem, consider the most common TaskRunner is Json.Net's Deserialize!

If my comments are wrong here, I apologize - but I have spent A LOT of time thinking about this lately.

[dsplaisted]

@jzabroski Exact version matching (and binding redirects to get around it) are required on .NET Framework, but not on .NET Core. So the issues you refer to (which were indeed painful) are mostly solved for .NET Core.