NuGetSdkResolver not able to find newly published NuGet package (due to http cache?)

[japj]

Details about Problem

VS2017 v15.8.1
TFS2018 Update3

We are building a MSBuild Sdk with a CI pipeline and publishing the NuGet to a TFS2018 PackageManagement feed.

We see that immediately after publishing the NuGet, the build servers do not seem to be able to pick up the new Sdk during a build.
MSBuild fails with "Error : Unable to find package with version".
Checking the Package Management feed shows that the NuGet actually exists.

After investigating, it looks like the NuGet http cache is causing this error, because if we wait 30 minutes or remove the NuGet cache folder the problem is solved (and the correct sdk is resolved).

Unfortunately there is no documentation on any way to configure the NuGetSdkResolver to not use the http cache (or a setting in NuGet.Config). There is a command line option though for NoCache or the RestoreNoCache MSBuild Property.

What can we do to help resolve this issue?

Detailed repro steps so we can see the same problem

1. configure nuget to connect to TFS feed
2. ensure http cache is refreshed
3. build new NuGet, publish to feed
4. try to consume the new nuget package (on a different machine)

[nkolev92]

@japj

The SDK resolver is just using NuGet's restore infrastructure under the hood. So there are no special switches that allow it to behave differently from regular restore.

The root cause is here; #3389.

There are a few options here:

• Keep it as it is, If needed the user can clear the http cache.

• Propagate RestoreNoCache to the SDK resolver.

• The SDK resolver could potentially not avoid the http caching, but it's a dirty workaround since it doesn't fix the common case.

//cc @rrelyea @livarcocc Thoughts?

[japj]

Ok, thanks for the information.
For now I “worked around it” by doing ‘nuget locals http-cache -clear’ before the actual build command in the build definition. At least until a better solution becomes available.

[nkolev92]

fyi @jeffkl

[jeffkl]

We'd want to fix it for the entire scenario for sure, ie we wouldn't want the restore of project SDKs to succeed but then actual package restore fails since its using the cache. This is a difficult problem to solve since we want to use the HTTP cache 99.9% of the time and still provide users a way to say that they want to refresh it. Maybe there should be an option where NuGet uses the cache and if the package isn't found locally it then falls back to hitting the feed? Otherwise people are just going to disable the cache 100% of the time (via nuget locals http-cache clear)

[nkolev92]

Maybe there should be an option where NuGet uses the cache and if the package isn't found locally it then falls back to hitting the feed

RestoreNoCache completely avoids the http cache, but there's no way to tell NuGet to refresh if it's not there, the context is long lost at that point.

Can you have a NuGet based resolver scenario where the version is not specified?
Because if there isn't, then propagating RestoreNoCache should be enough. The benefits of the http cache only exist in scenarios where the version is unknown or more than 1 gets downloaded.

[jeffkl]

Can you have a NuGet based resolver scenario where the version is not specified?

The SDK resolver will only do work if you pass a valid NuGet version without wildcards. Otherwise it does nothing. So we could plumb through RestoreNoCache as a global property but not as a property in a project or Directory.Build.props. I would need to update the MSBuild SDK resolver infrastructure to make global properties available.

[nkolev92]

What I'm thinking of is the resolver simply passing no cache anytime it needs to download a package.
Once a package is in the http cache, it's likely that it was already installed.

The only concern there would be having more than 1 version of the same sdk will lead to a network for each version.

[jeffkl]

If the user didn't specify RestoreNoCache and they just published an SDK and a package, then the SDK would restore but then the package wouldn't. I feel it would be better to have them opt into bypassing the cache for the whole restore since it would work as expected. But I'll trust your call in this case.

[nkolev92]

they just published an SDK and a package, then the SDK would restore but then the package wouldn't.

Yeah that's a valid concern. I think you are actually correct, let's avoid the confusion.

[jeffkl]

Closing as by design. If users want to avoid the issue, they'll need to manually clear their HTTP cache before restore. Also #3116 will fix this issue if its ever implemented.