When a ProjectReference and PackageReference in different TFMs have the same identity/version, Restore throws "An item with the same key has already been added. [...]"

[dagood]

Details about Problem

Full repro details at https://github.com/dagood/repro/tree/repro-restore-ref-same-key. I used the mcr.microsoft.com/dotnet/sdk:5.0 Docker image to minimally repro, but this also repros the same on my Windows machine.

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.100
 Commit:    5044b93829

Runtime Environment:
 OS Name:     debian
 OS Version:  10
 OS Platform: Linux
 RID:         debian.10-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.100/

Host (useful for support):
  Version: 5.0.0
  Commit:  cf258a14b7

.NET SDKs installed:
  5.0.100 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

System.Security.Cryptography.Cng.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <Version>5.0.0-rc.2.20475.5</Version>
  </PropertyGroup>
</Project>

System.Security.Cryptography.Pkcs.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>net5.0;netcoreapp3.1</TargetFrameworks>
  </PropertyGroup>
  <ItemGroup Condition="'$(TargetFramework)' == 'net5.0'">
    <ProjectReference Include="../System.Security.Cryptography.Cng/System.Security.Cryptography.Cng.csproj" />
  </ItemGroup>
  <ItemGroup Condition="'$(TargetFramework)' == 'netcoreapp3.1'">
    <PackageReference Include="System.Security.Cryptography.Cng" Version="5.0.0-rc.2.20475.5" />
  </ItemGroup>
</Project>

As far as I know I haven't seen this work before.

We hit it in source-build while trying to build dotnet/runtime: dotnet/source-build#1845 (comment)

This might not be supported--it doesn't make a whole lot of sense to me for a project to intentionally do this--but the error message should be improved at least! 😄 It took a while to figure out what the message was complaining about, work around it in source-build, and write this small repro.

Detailed repro steps so we can see the same problem

1. Clone https://github.com/dagood/repro/tree/repro-restore-ref-same-key
   (git clone -b repro-restore-ref-same-key https://github.com/dagood/repro && cd repro)

2. Run dotnet build ./System.Security.Cryptography.Pkcs/System.Security.Cryptography.Pkcs.csproj /bl

3. See this error:
   /usr/share/dotnet/sdk/5.0.100/NuGet.targets(131,5): error : An item with the same key has already been added. Key: (System.Security.Cryptography.Cng, 5.0.0-rc.2.20475.5) [/work/System.Security.Cryptograp hy.Pkcs/System.Security.Cryptography.Pkcs.csproj]

4. Open msbuild.binlog to see full stack trace:

System.ArgumentException: An item with the same key has already been added. Key: (System.Security.Cryptography.Cng, 5.0.0-rc.2.20475.5)
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey](List`1 source, Func`2 keySelector, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable`1 source, Func`2 keySelector)
   at NuGet.Commands.LockFileBuilder.CreateLockFile(LockFile previousLockFile, PackageSpec project, IEnumerable`1 targetGraphs, IReadOnlyList`1 localRepositories, RemoteWalkContext context)
   at NuGet.Commands.RestoreCommand.ExecuteAsync(CancellationToken token)
   at NuGet.Commands.RestoreRunner.ExecuteAsync(RestoreSummaryRequest summaryRequest, CancellationToken token)
   at NuGet.Commands.RestoreRunner.ExecuteAndCommitAsync(RestoreSummaryRequest summaryRequest, CancellationToken token)
   at NuGet.Commands.RestoreRunner.CompleteTaskAsync(List`1 restoreTasks)
   at NuGet.Commands.RestoreRunner.RunAsync(IEnumerable`1 restoreRequests, RestoreArgs restoreContext, CancellationToken token)
   at NuGet.Commands.RestoreRunner.RunAsync(RestoreArgs restoreContext, CancellationToken token)
   at NuGet.Build.Tasks.BuildTasksUtility.RestoreAsync(DependencyGraphSpec dependencyGraphSpec, Boolean interactive, Boolean recursive, Boolean noCache, Boolean ignoreFailedSources, Boolean disableParallel, Boolean force, Boolean forceEvaluate, Boolean hideWarningsAndErrors, Boolean restorePC, Boolean cleanupAssetsForUnsupportedProjects, ILogger log, CancellationToken cancellationToken)
   at NuGet.Build.Tasks.RestoreTask.ExecuteAsync(ILogger log)

Verbose Logs

Binlog: repro-restore-ref-same-key.zip

[nkolev92]

Known issue that @erdembayar is actively working on.
Maybe he can find the dup.

[erdembayar]

Known issue that @erdembayar is actively working on.
Maybe he can find the dup.

It's duplicate of Issue6795.
Draft PR for fix is here:
NuGet/NuGet.Client#3789

[dagood]

Looking at that PR, it seems like a different issue. It mentions "Prefer project reference over package reference, so remove the the package reference.". This isn't what I'd expect in this situation.

I would expect that (in this example) when building net5.0, it selects the project reference. When building netcoreapp3.1, it selects the package reference. That's based on these conditions:

  <ItemGroup Condition="'$(TargetFramework)' == 'net5.0'">
    <ProjectReference Include="../System.Security.Cryptography.Cng/System.Security.Cryptography.Cng.csproj" />
  </ItemGroup>
  <ItemGroup Condition="'$(TargetFramework)' == 'netcoreapp3.1'">
    <PackageReference Include="System.Security.Cryptography.Cng" Version="5.0.0-rc.2.20475.5" />
  </ItemGroup>

It sounds like that PR might actually make things worse by silently transforming the PackageReference into a ProjectReference when that isn't what you'd expect from the MSBuild code. 😕

[nkolev92]

It sounds like that PR might actually make things worse by silently transforming the PackageReference into a ProjectReference when that isn't what you'd expect from the MSBuild code

Project over package is the expectation for restore in general.

I think this scenario is slightly different from the linked one, and as such I am re-opening it.