Init Central NuGet Management Future Docs

[rconard]

Docs:

• Central Package Management
• Central Feed Management
• Central NuGet Maintainer

[anangaur]

We do not need Directory.Build.props. We would like this to be in Directory.Packages.props

[batzen]

The best, in my opinion, would be to automatically enable ManagePackageVersionsCentrally if Directory.packages.props is present.

[cwe1ss]

I also think that this should automatically be enabled once a Directory.packages.props-file exists.

[anangaur]

Nit: Can we have a simpler name for CentralManagementGroup?

[rconard]

I agree that it should be shorter, but I just don't have a good answer for what it should be.

I was just discussing this with @loic-sharma, and we walked through the process that I used to arrive at the current name.

If we use the simple term Group, that overloads a term already used in MSBuild. The user would put a Group in an ItemGroup.

If we use PackageGroup, then you are adding a PackageGroup to <PackageVersion>, <PackageReference>, and <PackageSource>.

I agree that it would be great if we had a short, descriptive term, but I haven't figured it out yet.

• Shorter than CentralManagementGroup
• Easy to search for on Bing/Google with the terms NuGet and MSBuild
• Doesn't mix concepts already in NuGet and MSBuild

[loic-sharma]

Other suggestions that I believe aren't used in MSBuild today: Scope, Area.

[cwe1ss]

If we use PackageGroup, then you are adding a PackageGroup to <PackageVersion>, <PackageReference>, and <PackageSource>.

TBH, I really like PackageGroup in combination with the other Package*-names. It's unique and concise and there's no need to invent new terms like "central" & "management". It's just an additional layer to manage packages.

Sure, this whole topic is about "centrally managing packages" but I don't think these terms should be reflected in code. Who knows, maybe one day Directory.packages.props might even be the default for new project/solution templates. If someone decides to use Directory.Packages.props, s/he will just be introduced to two simple new names: PackageVersion and PackageGroup.

Disclaimer: I'm not a native english speaker, so this just might sound good to me but not to native english speakers?!

[AArnott]

This section does not address a common problem we have with transitive dependencies, which is:

Let A depend on B.
Suppose the PackageVersion for A exists. B does not have a defined version.
I add a PackageReference to B.
I do not want to have to specify a version for B. It should implicitly come from the version specified by A.

Real world scenario: MessagePack depends on MessagePack.Annotations. Almost no one cares about this Annotations dependency nor references it explicitly. But if they do, they should get the one brought in by the main parent package with no further effort.
In particular I do not want to see MessagePack and MessagePack.Annotations both have to specify the version explicitly or else we have an update problem where someone updates the version for one and not the other and then we have version shear where either we get a bad combination of binaries or nuget restore fails because a dependency is accidentally lowered for a project when MessagePack.Annotations isn't updated but MessagePack is.

[rconard]

I'm not sure if I'm understanding this correctly, but this sounds like the case where no <PackageVersion> is specified for the transitive dependency.

This behavior already works as you described today, but I did not define it here in the document.

Would it help if I clarify that if no <PackageVersion> is defined and a single <PackageReference> is importing a transitive dependency that the version specified by the package for the transitive dependency is imported?

[AArnott]

Would it help if I clarify that if no <PackageVersion> is defined and a single <PackageReference> is importing a transitive dependency that the version specified by the package for the transitive dependency is imported?

I don't fully understand the second half of your question, and I'm not sure we're thinking of the same thing. What I'd like to see added to the docs (and be true in the product) is this:

When a PackageReference refers to a package for which no PackageVersion is defined, but that same package also appears as a transitive dependency of another package that does have a PackageVersion specified, then the version used for that PackageReference will implicitly come from the version required via the existing transitive dependency.

Or in other words (if this is easier to understand):

When PackageVersion for package ID A exists and A depends on B, a PackageReference may be added to A or B. If no PackageVersion for B is specified, the version for B will implicitly come from from the version required by A as expressed in A's dependencies.

[AArnott]

I've come to rely on Dependabot to keep our dependencies up to date in our github repos.
Presumably Dependabot will need to be taught this new syntax -- and until then, switching to CPVM will break the dependabot workflow.

[batzen]

Dependabot already understands the CPVM syntax and creates PRs with the correct changes. At least it works on my repositories.

[AArnott]

I wonder if Group would suffice, given the context is a PackageVersion item.

[AArnott]

An anti-pattern perhaps if the goal was to keep everything strictly top-level. But in repos that contain several disparate group of projects, allowing each sub-tree of projects adjust the package versions where necessary is a nice feature rather than an anti-pattern. So is this up to us to decide?
If we decide to have a tree of Directory.Packages.props files, will the IDE support that?

[cwe1ss]

Also, would it be a supported scenario to have a MySLN\Directory.Packages.props and a MySLN\ProductA\Directory.Packages.props and to call <Import Project="..\Directory.Packages.props" /> in that second file?

[AArnott]

These comments use versions that don't appear anywhere else. I don't understand what's being indicated here. Is it that we're explicitly downgrading the version dependencies? Such versions as these appear in multiple places elsewhere in the docs.

[AArnott]

I expected an error to result. I think it would be better to fail and require that the transitive dependency be made explicit where it can then name the group it comes from than come from the "first" item, which may feel random to the user if the project file itself only has one of the two items.

[cristinamanum]

I agree. If we rely on ordering we would expect that noop will be impacted by the order of the imports.

[AArnott]

Where in msbuild evaluation is the Directory.Packages.props file imported? If above this point (as its .props name suggests) then can't this be simplified to:

<PackageVersion Update="Newtonsoft.Json" Version="12.0.3" />

[AArnott]

I would more likely place the target framework conditions on the ItemGroup instead as it scales to cleaner xml. If the IDE was excellent at maintaining these as-is, I might leave them alone. :)

[tebeco]

Totally agree, the current syntax is very confusing
Here the package is the same and only the version change, this should NOT be in the csproj but in the Directory.Packages.props

With this example,

• the logic about Version stays in the file about CPVM
• It is grouped per TFM
• It's very similar as how MsBuild works
• the logic of "only reference this if ..." is kept in csproj
• less noisy / repeatition

Directory.Packages.props:

<Project>
  <ItemGroup Condition=" '$(TargetFramework)' == 'net472' ">
    <PackageVersion Include="foo" Version="1.0.0" />
    <PackageVersion Include="bar" Version="2.0.0" />
  </ItemGroup>

  <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0' ">
    <PackageVersion Include="foo" Version="5.0.0" />
  </ItemGroup>
</Project>

foo.csproj:

<Project>
  <PropertyGroup>
    <TargetFrameworks>net472;net5.0<TargetFrameworks>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="foo" />
  </ItemGroup>

  <ItemGroup Condition=" '$(TargetFramework)' == 'net472' ">
    <PackageReference Include="bar" />
  </ItemGroup>
</Project>

[tebeco]

also, the TargetFrameworks is evaluated latter on due to how PropertyGroup and ItemGoup are evaluated
given the fact that

• props file are supposed to be loaded / imported at the beginning of csproj files
• targets file are supposed to be loaded / imported at the end of csproj files

If the Directory.Packages.props is imported before the csproj and your project in the solution have a mix of TargetFrameworks and TargetFramework you can end up with TargetFramework being empty

Current Directory.Packages.props works almost well without group for Multi targeting to be honest ;)
There's just this little edge case that could be handled by moving to targets :)

[cwe1ss]

The example from @tebeco would be great for me as well. It would use existing MSBuild features and sounds like a more natural next step instead of having to use groups. It also naturally fits with how I structure multi-targeting project files right now - e.g. https://github.com/opentracing-contrib/csharp-netcore/blob/ebb38ade5d339c2e19d7c9a75297c82c1d4e6692/src/OpenTracing.Contrib.NetCore/OpenTracing.Contrib.NetCore.csproj

[AArnott]

nit: I would want the conditions placed on the property element itself.

[kevinchalet]

Hey guys,

As encouraged by @rconard, I took some time to read the whole specification and here's what I like about it...

• The "central" aspect is very well done, which is a good thing for a central management thing 😄
• The groups mechanism seems rather tooling-friendly, tho' it's still not clear yet how things would materialize in the NuGet UI.
• The transitive dependencies resolution is well documented.

Now, is it something I'd use personally? Not sure:

• It's powerful but also a bit complex (I suspect the learning curve will be complicated for some people).
• It's too verbose for multi-targeting scenarios:
  • You have to add a <PackageVersion>s for each TFM (fine).
  • You also need to add a group to each node.
  • Then, you need to bind each <PackageReference> to the correct group.
• You can already do even complex things with the existing CPVM bits and MSBuild conditions:

E.g to define per-TFM versions, you can use conditions in <ItemGroup>:

<ItemGroup Condition=" '$(TargetFramework)' == 'netcoreapp3.1'  ">
    <PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="3.1.8" />
</ItemGroup>

You can also override a package version for a specific project using something like:

<ItemGroup Condition=" '$(MSBuildProjectName)' == 'OpenIddict.Abstractions' ">
  <PackageVersion Update="Microsoft.Extensions.Primitives" Version="2.1.6" />
</ItemGroup>

Supporting hierarchy with Directory.Packages.props could also be useful to address that, tho' I agree it's no longer strictly "central" in this case.

Overall, my feeling is that this new specification adds a new (complex) layer on top of things that are already supported using MSBuild conditions, but I may be wrong.

There's also something important I didn't see in the specification: update ranges. If I have multiple <PackageVersion>s pointing to the same package, how will the NuGet PMUI (or automated tools like Dependabot) know what range is acceptable when suggesting potential updates? E.g if I have a netcoreapp2.1 TFM and reference Microsoft.Extensions.Primitives 2.x, I'll likely don't want PMUI to update it to 3.x.

Note: we can already use ranges in Version, but these values are also used to produce .nupkgs and you may not want the "update ranges" to leak in this case.

[batzen]

Please keep path length limitations on Windows in mind.
This might escalate quickly.

[rconard]

This is an excellent point.

From what I can tell, it is now much easier to enable NTFS long file name support for long path names. If we detect that long path names are not enabled on a machine, I believe that we should be able to prompt the user (on a development machine) or automatically update the registry key (in a build environment). In many cases, this will already be enabled.

Since development and build machines are usually running an updated version of Windows 10 client, Windows Server, or Linux, this should solve the path length limitations for a very high percentage of users.

I would be interested to hear more feedback on this, but it seems like enabling NTFS long file names once to improve cache organization and security is a worthwhile tradeoff for the more experienced users who need Central NuGet Management.

[rconard]

• It's too verbose for multi-targeting scenarios:

  • You have to add a <PackageVersion>s for each TFM (fine).
  • You also need to add a group to each node.
  • Then, you need to bind each <PackageReference> to the correct group.

• You can already do even complex things with the existing CPVM bits and MSBuild conditions:

@kevinchalet, what would you think about making TFM a first-class attribute?

<ItemGroup>
    <PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="3.1.8" TFM="netcoreapp3.1" />
</ItemGroup>

There are advantages and disadvantages to this approach, and I am curious what everyone thinks.

[tebeco]

I would love to have something less noisy, and if possible relly on MsBuild Property by default in fs/csproj:
From:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <PackageReference Include="Newtonsoft.Json" CentralManagementGroup="A" />
        <PackageReference Include="foo" CentralManagementGroup="A" />
        <PackageReference Include="qux" CentralManagementGroup="A" />
        <PackageReference Include="xyzzy" CentralManagementGroup="A" />
        <PackageReference Include="bar" CentralManagementGroup="A" />
    </ItemGroup>
</Project>

to something like

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <PropertyGroup>
        <CentralManagementGroup>A</CentralManagementGroup>
    </PropertyGroup>
    <ItemGroup>
        <PackageReference Include="Newtonsoft.Json" />
        <PackageReference Include="foo" />
        <PackageReference Include="qux" />
        <PackageReference Include="xyzzy" />
        <PackageReference Include="bar" />
    </ItemGroup>
</Project>

[AArnott]

FWIW if they do go with the per-reference approach, you can still be less noisy with this syntax:

<Project>
    <ItemDefinitionGroup>
        <PackageReference>
            <CentralManagementGroup>A</CentralManagementGroup>
        </PackageReference>
    </ItemDefinitionGroup>
    <ItemGroup>
        <PackageReference Include="Newtonsoft.Json" />
        <PackageReference Include="foo" />
        <PackageReference Include="qux" />
        <PackageReference Include="xyzzy" />
        <PackageReference Include="bar" />
    </ItemGroup>
</Project>

[tebeco]

Is there a way to make it work with condition being evaluated ?
This will be a bit straight forward to understand the condition between multiple groups
And this will also allow consumer to group all package together for the same group, which make a bit more sense when looking for stuff

Though that won't the consumer to see that it's also used elsewhere

TBH, I've not seen multiple group beeing used in the same project, that feels very weird.
If it's used for Multi targettings, then this already works without groups in fact
MsBuild targets works well (I confirmed I already used CPVM with target nicelly without groups)

From :

    <ItemGroup>
        <PackageVersion Include="foo" Version="3.3.3" CentralManagementGroup="A" />
        <PackageVersion Include="bar" Version="3.0.0" CentralManagementGroup="A" />
        <PackageVersion Include="foo" Version="30.0.0" CentralManagementGroup="B" />
        <PackageVersion Include="bar" Version="30.0.0" CentralManagementGroup="B" />
    </ItemGroup>

to something like :

    <ItemGroup Condition=" '$(CentralManagementGroup)' == 'A'">
        <PackageVersion Include="foobar" Version="12.0.0" />
    </ItemGroup>

    <ItemGroup Condition=" '$(CentralManagementGroup)' == 'A'">
        <PackageVersion Include="foo" Version="3.3.3" />
        <PackageVersion Include="bar" Version="3.0.0" />
    </ItemGroup>

    <ItemGroup Condition=" '$(CentralManagementGroup)' == 'B'">
        <PackageVersion Include="foo" Version="30.0.0" />
        <PackageVersion Include="bar" Version="30.0.0" />
    </ItemGroup>

[tebeco]

As a side note on Group, I would very much love to avoid group to handle multi targeting.
This will add an extra complexity to it, and will drive away people from it.

As of today using Directory.Build.targets works sort of the same as CPMV, and even this syntax (see this example) seems to be complicated to explain because not everyone knows msbuild.

After using paket for ~3-4 in production (team of 30-40 dev) I also noticed that the basic notion of "direct vs transitive" is not something everyone understand properly,
They sort of know what is a dependency of a dependency, but not the "word itseld"
Also, it's not always straight forward when we ask to stop explicitly referencing transitive (...it's transitive), as it can have VERY BAD side effect on the "Tree" resolution

And now that i've come down to this multiple times, I've discovered that it's definitly not clear for the majority that when you think dependencies, you have to strat from the "runnable project" (main + each test + ....) because that's the one referencing the rest of the projets ...

All of this to say ... if we can remove the Group but just make sure the Versions are computed by respecting targets import order (at the end of the proj) and not props import order (beginning of the proj files)
That would make CPMV works for multi targetting without the needs of group, hopefully

Altermative

Each TFM is an implicit group, and nobody has to declare it ?

[tebeco]

as seen here:
#1 (comment)

<ItemGroup>
    <PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="3.1.8" TFM="netcoreapp3.1" />
</ItemGroup>

multiple questions:

• can TFM be something already existing in MsBuild to avoid a "new thing" (like TargetFramework or something else existing)
• will it handle properly the resolved TFM especially since net5.0 changes

eg:
<TargetFramework>net5.0</TargetFramework> will output net5.0-windows7
I supposed that will all the list of existing TFM, the fact that we can cross target and the change on TFM that can OR NOT include a platform version this needs to be triple checked ^^

eg:
What if you want (I understand group could also be able to do that, but I'd love to avoid group for XTarget ^^)

• foo version 1.2 for net5.0
• foo version 1.0 for net5.0-ios14

[tebeco]

Will it be part of dotnet nuget locals --list all ? so that the --clear will also apply
eg:
dotnet nuget locals --clear contoso

[tebeco]

Due to the way VM are created:

• OS Disk
• Data Disk

We use EnvVar to change these cache location today on the CI, because the Os Disk is small and we can't predict when it will be critical.
This might help scenario like Azure Disk being mounted to re-use cache <== not sure if this is a good idea ^^

[tebeco]

That would be the PERFECT place and opportunity to do that :
<PackageSource key="NuGet.org" Feed="https://nuget.org/api/v3/" Proxy="http://corpo:8080" ProxyAuthentication="NTLMv2|windowscred|networkcredential......" />

[tebeco]

The image sounds like somethings I don't expect to see happening at all.
If groups are here to isolated dependency tree, I was love to see resolution fails by default rather than picking it from another group

This will lead to headake when trying to understand why downgrade can happen.

If you still think this could be beneficial to someone, I'd rather hint to the user that this could not be resolved because etc ...

Group should probably be fully isolated by default, but if you want to to ignore this I would love an OPT_IN "yes you can go pick in global/or this group/"
Just like the Scope of the PackageSource

[tebeco]

I guess the SHA is realated to the Feed or Group, is it pertinent to explain what it is or to get a CLI command to list thing and assiociate them ?

[tebeco]

Is it possible to have a flag to allow disabling the cache completly on a PackageSource ?
Why ?
this example is perfect, when you build your own nuget and try to consume is on a project, you don't always bump the version, so you always generate 20.0.0
Usually you just do

• code change
• build
• pack on build enabled
• output package path go to c:\installed_packages
• you run dotnet restore else where

As of today the this will not bump the version
then dotnet restore will mostly no-op and use older build

[AArnott]

It's a good use case, but IMO it's out of scope of this spec.
FWIW I solve this in either of two ways, usually:

1. blow away the %userprofile%\.nuget\packages\mypackage\1.0.0 folder each time I want to restore a new version.
2. ensure the package version changes each time, using nerdbank.gitversioning and making sure each thing I test is a new commit (whether novel or amended) so that the commit ID changes. NB.GV stamps each package with a version that includes the commit ID when working in topic branches to guarantee uniqueness.

[cristinamanum]

1. What would happen if the qux is not in Group A but global? Will the version from global be used?
2. What would happen if bar is not in Group A but global? Conform with the fallback rule its version will be from the global group. Will the qux Group A version be used?
3. We could imagine more transitive dependencies jumping between Global and Group A will picking the version from Global or Group A be ok?

[cwe1ss]

After reading through the specs I pretty much entirely agree with @kevinchalet's summary #1 (comment)

For our scenarios, I'd also prefer to directly target TFMs by just using the existing Condition="'$(TargetFramework)' == 'net5.0'"-syntax as that would leave the project files untouched. This would also make it easier to e.g. slowly transition some projects from net5.0 to net6.0.

If that would work, we'd probably not need "management groups" for our scenarios - our solutions are not big/complex enough right now. But of course, I can definitely see scenarios where "management groups" do make a lot of sense!