Allow users to determine package resolution strategy during package restore - direct or transitive

[Wil73]

I am not going to debate if highest or lowest should be the default: leave the actual behavior up to the consumer.

Why on earth should Microsoft FORCE users to adopt the "safest is best" strategy in the first place?

Is there any reason why we cannot add a switch to nuget.exe and NPM that allows the user to decide their own package restore strategy?

This way consumers of nuget can align nuget behavior to their own business strategies, be they conservative or aggressive.

What I don't understand is why Microsoft has to decide this for consumers in the first place.

To me this is a straight up consumer decision based on private business strategy and policy.

It makes absolutely no sense why a tool vendor should allow themselves to unilaterally decide business strategy for all nuget consumers on the planet without an option to override that.

It's been two years and there's no apparent movement on this issue.

Leave lowest version as the default strategy if you want Microsoft, but for goodness sakes offer a switch to allow consumers to make their own business decisions.

[davidfowl]

I am not going to debate if highest or lowest should be the default: leave the actual behavior up to the consumer.

Lowest should be the default, but that's just my opinion. It's the best way to get a "working set of packages". Everyone in the graph gets what they compiled against and things only roll forward minimally if sibling packages need higher versions.

Why on earth should Microsoft FORCE users to adopt the "safest is best" strategy in the first place?

It's more of a "safest is default strategy" because people want to get a working set of packages by default. In a system where you need binary compatibility between components to in order to make things function things have a better chance of working if you stay closer to what components build, and test against.

Is there any reason why we cannot add a switch to nuget.exe and NPM that allows the user to decide their own package restore strategy?

This is already possible when using packages.config based projects by picking the dependency behavior on install:

We don't own NPM, that's the node package manager, not sure if you mean something else.

To properly do this with PackageReference, a lock file needs to be introduced (which has been discussed anyways).

[Wil73]

I am not going to debate what the default ought to be. I really am not concerned over that. Technically this is NOT a default when you can't configure it otherwise. It's only a default if you can change it. We can't.

My company is frustrated that Microsoft will not allow businesses to DECIDE Nuget resolution behavior for themselves.

It makes absolutely no sense why Microsoft should have a monopoly on the business policies of the private sector: this is in effect what Microsoft does here.

Make package resolution configurable via a simple switch.

By NPM I mean NuGet Package manager in Visual Studio.

I don't see why any lock file is required. Just add an additional switch in nuget.exe and Nuget Package Manager that indicates if lowest, highest etc should be the expected behavior.

Package.Config is for NuGet 1 and 2. That's well deprecated by now.

This is clearly a business decision that Microsoft currently has constrained to Microsoft to decide.

There are many switches in Nuget already. I don't understand why we cannot simply add one more.

I fail to understand why it takes over two years for Microsoft to resolve this situation. The current work around is to explicitly declare EVERY dependency in package references (for Nuget 4) and in Project.Json (for Nuget 3). This is a clumsy and labor intensive work around.

Just add a darn switch to indicate Nuget's expected resolution behavior so users can manage it themselves.

It makes no sense why Microsoft must control this in the first place.

Just add a switch.

[davidfowl]

I don't see why any lock file is required. Just add an additional switch in nuget.exe and Nuget Package Manager that indicates if lowest, highest etc should be the expected behavior.

It's required with PackageReference because there's no way to make restore deterministic if you don't add it. If there's no asset you can commit to source control to lock dependencies then restoring the latest versions of all dependencies could give dramatically different results build to build. That might not be an issue if you tightly control the feeds but it is in the general case so just adding a switch isn't sufficient for the masses. This feature needs to be paired with a lock file to make it sane to use for the majority of customers.

[Wil73]

I think having the flexibility to choose resolution strategies is just as important as having a deterministic build. I do agree a deterministic build is important, but so is flexibility, which has been in short supply for long enough.

There is so much debate as to what the restore default should be (highest or lowest), but the point is irrelevant: higher or lower should be a choice the business makes, not Microsoft.

If I want a predictable build I can always just stick with project.json, check in my project.lock.json file with lock set to true, or just use fixed version numbers (the latter is the most predictable way, but the least flexible and most labor intensive). A package can get de-listed at any moment, so defaulting to lowest version does not guarantee a predictable build output either.

Microsoft needs to add switches to allow business to define their own resolution defaults. Microsoft has no business making these decisions for businesses in the private sector.

[davidfowl]

There is so much debate as to what the restore default should be (highest or lowest), but the point is irrelevant: higher or lower should be a choice the business makes, not Microsoft.

Sure. Like I said, it's possible with packages.config and that functionality should be ported to PackageReference (project.json is pretty much deprecated at this point) along with a way to lock the graph after resolution.

I think having the flexibility to choose resolution strategies is just as important as having a deterministic build. I do agree a deterministic build is important, but so is flexibility, which has been in short supply for long enough.

I think you can have both though. IMO it wouldn't be great if we had a feature that most people couldn't use because we didn't think through the end to end.

If I want a predictable build I can always just stick with project.json, check in my project.lock.json file with lock set to true,

That doesn't work. Locked: true isn't supported anymore (hasn't been for a while).

or just use fixed version numbers (the latter is the most predictable way, but the least flexible and most labor intensive).

Fixed versions of the entire dependency graph would work, but it's extremely labor intensive.

A package can get de-listed at any moment, so defaulting to lowest version does not guarantee a predictable build output either.

I believe this isn't respected when resolving dependencies in project.json so it is deterministic.

[Wil73]

project.json will resolve to the lowest version it can find. If the version the build uses normally gets de-listed your deterministic build is out the window.

If you are going to use dependencies from nuget.org or any other third party site, you cannot get a deterministic build every time, even with nuget restore working as it does now.

When you depend on a third party for some of your libraries you cannot determine what your build will look like every time no.

The alternative is to host any libraries you use locally so you have complete control over their list status.

I think we both agree package.config with fixed version numbers is far too labor intensive.

Project.json is far more flexible than package.config. Package.config is even more deprecated and out of date than project.json.

We still use project.json where I work along with Nuget 3.5 since Nuget 4 does not offer any more flexibility with regards to package resolution options.

When Microsoft smartens up and lets go of its monopoly on package resolution strategy logic we will adopt a later version.

As it stands now Nuget 4 does not offer anything that Nuget 3.5 doesn't already do for us. Locked: true does work for us with Nuget 3.5. Though we don't want fixed version numbers. We can update a library quite regularly. We need flexibility here more than we need predictability.

We use TFS automated builds with a fully featured Artifact Server here as well. If we need any build of any configuration we can pull that from TFS in just seconds. Flexible builds are a more pressing need for my company than deterministic builds.

The Lower version is no guarantee of safer. If that was the case we would all still be running Windows 95.

[davidfowl]

project.json will resolve to the lowest version it can find. If the version the build uses normally gets de-listed your deterministic build is out the window.

1. It's super uncommon to de-list things. That said, I'm not even sure restore respects this.
2. If you delete things, your build is broken anyways.

If you are going to use dependencies from nuget.org or any other third party site, you cannot get a deterministic build every time, even with nuget restore working as it does now.

You can 98% of the time (I made that up), because de-listing on nuget.org is rare and deleting is impossible. Now, if you're mirroring your dependencies to a myget feed or controlled feed, it's even better since you'd rarely delete or de-list anything that was being depended on.

However, de-listing/deleting is rare, if we support resolving the latest dependency version without a sane story to lock it, we're turning the 2% case into the 98% case.

When you depend on a third party for some of your libraries you cannot determine what your build will look like every time no.

Not following this argument.

Project.json is far more flexible than package.config. Package.config is even more deprecated and out of date than project.json.

We still use project.json where I work along with Nuget 3.5 since Nuget 4 does not offer any more flexibility with regards to package resolution options.

Hopefully you'll switch to PackageReference since that's where all of our energy is at the moment. Good to know locked: true still works in the version of the client that you're using.

All of that said I agree with you, I just think when we do that feature it'll be accompanied with a lock file so that things remain deterministic. That shouldn't be too hard to accomplish regardless.

[Wil73]

I would have to double check for Nuget 4, but you can get a deterministic build using highest version with a version number limit. I don't think Nuget 4 will allow this scenario out of the box either. It's not 100% predictable, but the point is we cannot even configure that.

If you want to lock, then use VS 2015 and Nuget 3.5. You just have to check in your project.lock.json.
That works just fine. I haven't investigated locking for Nuget 4 and VS 2017, but it sounds like Microsoft dropped the ball yet again.

I like the high level design of NuGet, but the implementation is dreadful on many levels. MS has had over five years now to get NuGet to a sensible level and it's still not there.

If locks are out the window with Nuget 4, and Nuget 4 STILL won't offer a switch for businesses to configure their own business policies how is Nuget 4 any better?

Moving references to the project file accomplishes what? That work effort could have been put into offering what businesses need: configurable restore policies and restore locks. I don't recall seeing a single post of anyone complaining that the project.json file format urgently needs a renovation.

There are hundreds (I haven't counted, but I stopped a long time ago) of people complaining about the inflexible lowest is best logic hard coded into Nuget right now.

On the face of it, it looks like MS wants everyone to use the latest version of their software, but the oldest version of everyone else's software. I find it ironic how aggressively Microsoft endorses adoption of new releases of their products, but Nuget logic has a "lowest is safer" constraint hard coded into the binary.

If lowest was safer we would all still be running Windows 95.

[davidfowl]

I would have to double check for Nuget 4, but you can get a deterministic build using highest version with a version number limit. I don't think Nuget 4 will allow this scenario out of the box either. It's not 100% predictable, but the point is we cannot even configure that.

Not only is it not predictable, it's probably more broken (especially for dependencies out of your control).

If you want to lock, then use VS 2015 and Nuget 3.5. You just have to check in your project.lock.json.
That works just fine.

VS 2017 with PackageReference is the future of nuget and it needs a proper lock file story.

[Wil73]

How is getting the latest version "out of control?"

Our approach at our company is to use a fixed version number for third party dependencies and use the highest version for in house packages. We don't update third party dependencies nearly as often as we update internal packages. We have a very large in house class library framework that relies on nuget to be sensible: right now it's not. We can make nuget sensible, but we have to hack project.json to make that happen.

This is not out of control. Our dependencies are quite stable. We have a solid change management process and automated testing. Adopting a new version when that version gets released does not make a process out of control. It just makes your company agile and aggressive in adopting the new.

This goes back to my original point: this is a controversial debate left to the business to decide, not a tool vendor.

I cannot say for any given company if lower or higher version resolution strategy is better. I think it depends on the company and how aggressively they want to adopt new tools. Again this is such a controversial topic it is absurd to have a tool vendor make these decisions for everyone and hard code those policies into their binaries.

That being said, just offer a darn switch. That ends all debate on the matter. lol

[davidfowl]

How is getting the latest version "out of control?"

You misread, I said "dependencies that are out of your control".

We don't update third party dependencies nearly as often as we update internal packages.

Sure. How do you restrict the resolution strategy to only update certain dependencies?

This goes back to my original point: this is a controversial debate left to the business to decide, not a tool vendor.

I think I agreed with you each time you made this comment but my position hasn't changed when it comes to needing a lock file with this feature.

That being said, just offer a darn switch. That ends all debate on the matter. lol

I have little say in what the nuget team does here so I can't say how or when this would ever happen. I'm guessing it won't work on VS 2015 with project.json though (I could be wrong). I'm also guessing this mode needs to be source controllable (probably in nuget.config) since you'd want command line restore and VS restore to do the same thing.

[Wil73]

You can use project.json syntax to restrict to specific versions or simply de-list other packages so you only have one version in your package line. It's a hack, but it works.

Yes, you would want a switch or option for both NPM and the command line restore of course.

[anangaur]

That's a long thread over the weekend. :)
From what I have observed (here - this thread as well as talking to other folks), the requirements seem to be the following:

1. Allow users to decide the dependency resolution strategy - direct as well as transitive
2. Allow users to lock down on the dependency versions in the transitive world - to enable repeatable build

Both # 1 and # 2 seem to be lacking with PackageReference. # 2 seems like already working with lowest is the best strategy but will falter with # 1 coming into the fore. So I think NuGet will have to solve both together atleast for PackageReference.