Issues with .NET Standard 2.0 with .NET Framework & NuGet

[terrajobst]

Summary

We've designed .NET Standard & our tooling so that projects targeting .NET Framework 4.6.1 can consume NuGet packages & projects targeting .NET Standard 2.0 or earlier. Unfortunately, we've seen a few issues around that scenario. The purpose of this document is to summarize the issues, outline our plan on addressing them, and providing workarounds you can deploy with today's state of our tooling.

Symptoms and root cause

The primary symptom is that applications crash with a FileLoadException or a FileNotFoundException. Another symptom is warnings at build time regarding assembly versions. This is due to one or both of the following issues:

1. Missing binding redirects
2. Missing binaries that come from indirect NuGet packages

Missing binding redirects

.NET Standard 1.x was based around contracts. Many of these contracts shipped with .NET Framework 4.5 and later. However, different versions of .NET Framework picked up different versions of these contracts, as by-design of contract versioning. As a side effect of marking .NET Framework 4.6.1 as implementing .NET Standard 2.0, some projects will now start picking up binaries built for .NET Standard 1.5 and 1.6 (as opposed to previously where .NET Framework 4.6.1 was considered as implementing .NET Standard 1.4). This results in mismatches of the assembly versions between what was shipped in .NET Framework and what was part of .NET Standard 1.5/1.6.

This can be addressed by binding redirects. As writing them by hand sucks, we added an Automatic Binding Redirect Generation feature in .NET Framework 4.5.1. This feature is opt-in. Unfortunately, it's not enabled based on target framework, but by which target framework was selected when the project was created (as the feature is turned on via an MSBuild property that is conditionally emitted by the template). In practice, this means it's mostly off if you often upgrade existing projects, rather than creating new ones.

Missing binaries

There are two primary flavors of NuGet: packages.config and PackageReference.

• With packages.config, each project has a config file with a flattened graph of all the NuGet dependencies. The project file in turn has direct links to all the assets. The assets are selected at install time. None of this includes indirect NuGet references coming from referenced projects.

• With PackageReference each project contains MSBuild PackageReference items. The project file contains no references to any assets as the assets are selected at build time. Package restore will compute the graph of all packages, including indirect NuGet references coming from referenced projects.

The default for .NET Framework projects is packages.config. This ensures more compatibility because PackageReference doesn't support all the features that packages.config did, for example, PowerShell install scripts and content.

The only supported mode for SDK-style projects (.NET Core/.NET Standard) is PackageReference. This means that a .NET Framework project referencing a .NET Standard project ends up crossing the streams between two different NuGet models. When the .NET Standard project references NuGet packages that the .NET Framework project doesn't reference, the application ends up missing all binaries coming from those packages.

Why has this worked before? Because with packages.config, all dependencies are copied to each project's output folder. MSBuild copies them up from there. With PackageReference, we don't copy the binaries because it relies on the consuming project to see its dependencies and extract the proper asset itself. This allows the consuming project to pick up the right assets for packages that use bait & switch (which many of the .NET packages must do).

Plan

The plan is to address these issues moving forward as follows:

1. Converge on PackageReference for all project types, including .NET Framework. The short-term plan for (1) is to start blocking project-to-project references in Visual Studio 15.4 that will end up crossing the streams between packages.config and PackageReference. This block is UI only; you can still edit the reference by editing the project by hand. The error message will instruct you to switch the .NET Framework project to PackageReference if you want to reference a .NET Standard project. Referencing .NET Standard binaries or NuGet packages will not require this, it's only about project-to-project references. In later releases, we plan on providing a converter. The challenge is that packages.config has features we can't offer for PackagReference across the board, in particular PowerShell install scripts and content. We'll need good guidance and mitigations, if applicable.

2. Ensure binding redirects are on by default. Short term, this means we need to fix our target files to make sure we turn on automatic binding redirect generation. However, binding redirects don't work well in all scenarios, when there is no application project (like unit tests or add-ins). We need to work on a plan to bring the regular “higher wins” binding policy without binding redirects. This needs a proposal and proper vetting, but it seems we've reached the point where this is necessary.

Workarounds

Regular .NET Framework projects

1. Enable automatic binding redirects in the root .NET Framework application.
2. Make sure your root application project doesn't use packages.config but uses PackageReference for NuGet packages
   • If you currently don't have packages.config, simply add <RestoreProjectStyle>PackageReference</RestoreProjectStyle> to your project file
   • If you currently do have a packages.config, convert the contents to packages references in the project file. The syntax is like this:
     • <PackageReference Include="package-id" Version="package-version" />

ASP.NET web applications and web sites

1. Web applications and web sites don't support automatic binding redirect generation. In order to resolve binding conflicts, you need to double click the warning in the error list and Visual Studio will add them to your web.config file.
2. In web application projects, you should enable PackageReference like mentioned above. In web sites, you cannot use PackageReference as there is no project file. In that case, you need to install all NuGet packages into your web site that any of the direct or indirect project references depend on.

Unit tests projects

By default, binding redirects aren't added to class library projects. This is problematic for unit testing projects as they are essentially like apps. So in addition to what's outlined in automatic binding redirects you also need to specify GenerateBindingRedirectsOutputType:

<PropertyGroup>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
</PropertyGroup>

[mungojam]

Thanks for this, things do seem a little unpolished, so it's useful to have some clarity.

Other things that currently make it painful for us to mix .net framework with .net standard are a FileNotFoundException that will make it difficult for us to migrate our internal packages in a safe way: NuGet/Home#5715 and the lack of visibility of indirect packages when switching to PackageReferences: UserVoice to convert all projects to new common project format

[mungojam]

Is it correct that PackageReference is not supported for anything other than .net core, .net standard and UWP? See the documentation:

At present, package references are supported in Visual Studio 2017 only, for .NET Core projects, .NET Standard projects, and UWP projects targeting Windows 10 Build 15063 (Creators Update).

Hopefully that will change in future as it is an option in Visual Studio and is the workaround that you describe.

[0x53A]

Hi @terrajobst, is there a document detailing which scenarios are supported for .NET Framework 4.6.1 and .NET Standard 1.6?

I created a net461 old-sdk project in VS2017.3 and added a .NET Standard 1.6 NuGet. The build automagically copied the required shims to the output, totalling 100 files. ✅

If I open the exact same project in VS2015 and build, the output contains only 4 files, so the shims are missing. ❌

---

Where do the shims come from, anyway? I was following some earlier discussions, and I know at first they were deployed via a NuGet package. But later discussions talked about some machine-wide installation. What was the final result? What do we need to deploy to a build server to enable this scenario?

[terrajobst]

@mungojam

Is it correct that PackageReference is not supported for anything other than .net core, .net standard and UWP?

You can use PackageReference in .NET Framework projects, but as I outlined there is a loss in certain features. For instance, if you install packages that rely on PowerShell scripts (for instance, EF6) they will no longer work. Also, packages that require content (i.e. source files that are copied to the consuming project) they will also no work.

@0x53A

Which NuGet package are you consuming? For .NET Standard 1.x, the package is supposed to depend on NETStandard.Library which will bring in the shims.

[0x53A]

@terrajobst repro is here: https://github.com/0x53A/n16test

The nupkg was created calling "dotnet pack" on
https://github.com/0x53A/n16test/blob/75fcfdc0d540531c30b805e74bb239e42d13ffea/ns16lib/ns16lib.csproj#L1-L7.

The resulting package looks like this:

To reproduce my issue:

1. open https://github.com/0x53A/n16test/blob/master/net461exe/net461exe/net461exe.sln in VS 2017.3 and build.
2. note that /bin/ contains 100 files.
3. clean /bin/, re-open in VS 2015.3 and build
4. note that /bin/ contains only 4 files.

[FransBouma]

In vs2015, in a csproj targeting v4.6.2, I wanted to add a reference to Entity Framework Core 2.0, but that failed with this error:

Attempting to gather dependency information for package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' with respect to project 'EF Core 2\NWEFCore2.Persistence', targeting '.NETFramework,Version=v4.6.2'
Gathering dependency information took 0,87 ms
Attempting to resolve dependencies for package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0'
Resolved actions to install package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0'
Retrieving package 'Microsoft.EntityFrameworkCore.SqlServer 2.0.0' from 'nuget.org'.
Install failed. Rolling back...
Package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' does not exist in project 'NWEFCore2.Persistence'
Package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' does not exist in folder 'C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.3\UnitTests\EntityFramework\packages'
Executing nuget actions took 62,07 ms
install-package : Could not install package 'Microsoft.EntityFrameworkCore.SqlServer 2.0.0'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.6.2', but the package does not contain any assembly references or content fi
les that are compatible with that framework. For more information, contact the package author.
At line:1 char:1
+ install-package Microsoft.EntityFrameworkCore.SqlServer
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Install-Package], Exception
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.InstallPackageCommand

Likely a bug in the Nuget client in 2015 (but I am under the assumption I have the latest version installed), but IMHO this should simply work as .NET 4.6.2 supports .NET standard 2.0

(edit) This does work in vs2017 15.3.x. That's not an excuse to not fix this! Many devs are on 2015 and have no plans to move to 2017 (yet) and as .net standard 2.0 is supported on .net 4.6.2, they should be able to expect this to just work.

I don't know if this is the right location to file this (likely not!), if not please point me to the repo where I should file this, thanks.

[isaacabraham]

cc @forki I wonder what impact this has on Paket vis-a-vis automatic binding redirects (Paket has the option to generate BRs on demand based on the dependency graph).

@terrajobst If there's a BR already in the app.config file and you turn on the automatic BR redirection, who wins?

[forki]

@isaacabraham I have no idea. this will bring interesting new problems to us.

[FransBouma]

And another problem:

Using EF Core 2.0 in a .NET 4.6.2 project: as soon as you create a DbContext derived type, you'll get this error:

System.IO.FileLoadException : Could not load file or assembly 'System.ValueTuple, Version=0.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
  Exception doesn't have a stacktrace

Likely caused by the same issue as the ValueTuple containing assembly isn't there. This is in vs2017 15.3.x btw.

Brb, throwing some heavy furniture against the wall screaming.

[0x53A]

@FransBouma This looks familiar: #476 (comment)

[matthid]

How about dropping this scenario and make it work "cleanly" with a future version instead of trying to get with the head through the wall?

[dasMulli]

Note that the binding redirection documentation is missing the critical

<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>

property which is crucial to making "classic" test projects work (as well as libraries loaded via reflection).

[FransBouma]

@0x53A yes, I think that's similar. The bindingredirect however kills nunit for some reason (the test is now simply not seen by it and not run).

@dasMulli Thanks for that, but I am at a loss where to add that... adding it to the bindingredirect nor the csproj etc. does make nunit work again.

And here I am hoping that after weeks of fighting with MS pre-alpha quality tooling (Where's the QA testing? We paid for this software!), this week would go smoothly. Nope

(edit) fsck it, converting to new csproj format, perhaps that gives some results.

(edit) for ref: dotnet/sdk#1070 adding the 2 redirects made nunit work again. With the new csproj format the bindingredirect isn't needed, the valuetuple issue isn't popping up, but the attribute referred by @dasMulli are required to make things work.

Man... one could start wondering what 'alpha' quality means these days if 'RTM' is this quality.