Add option to disable NuGet Cache

[vsfeedback]

Migrated from Visual Studio IDE UserVoice forum

Once you have a lot of packages (we have over 200!) The NuGet cache becomes a complete pain. It continually locks files requireing VS restarts, it gets out of date with dependencies causing package restore to fail. These things waste hours of time... Can we have the option to just update projects directly from the package sources rather than using the VS cache?

This issue has been moved from https://developercommunity.visualstudio.com/content/idea/351752/add-option-to-disable-nuget-cache.html
VSTS ticketId: 700852
These are the original issue comments:

Fiona Niu[MSFT] on 10/9/2018, 11:56 PM (27 days ago):

Thank you for taking the time to provide your suggestion. We will do some preliminary checks to make sure we can proceed further. You will hear from us in about two weeks on our next steps.

[nkolev92]

The global packages folder copy of the package is the only copy in PackageReference, which means this is not feasible.

Furthermore the assemblies being locked are not because of NuGet, but rather other processes consuming those packages.

[jainaashish]

Not sure what is the ask here. but there is already option to overwrite global packages folder which essentially allows you to maintain solution specific packages folder. What else is needed here?

[cilerler]

I just hit the same kind of issue, and I am wondering if there is a way to mark a package not to get cached?

In my case, I always update the package in my local and had to delete it from ~.nuget\packages so the application that takes the package can get the latest and the greatest.

[zivkan]

@cilerler as the first reply says, the global packages folder is used as "the location" for PackageReference projects. The NuGet team does not refer to the global packages folder as a cache for multiple reasons, and this is one.

What this means is that the compiler reads assemblies from the global packages folder. MSBuild copies these assemblies from here to your project's output folder. During a build, it is "the source of truth" and usually the only location on the computer where the package exists. If NuGet doesn't extract packages to the global packages folder, then your project/the build can't use the package.

NuGet's docs need to be clearer, but one of the critical design choices/assumptions is that packages are immutable. This is important so that NuGet knows that once a package is downloaded and extracted on the local machine, it does not need to query remote sources to see if the package has changed.

This creates challenges for testing. My preferred way to mitigate this is to run tests in a test directory where a nuget.config file is created that redirects the global packages folder that is cleaned every test. You can hard code the nuget.config file contents, or use your favorite XML library (if the nuget.config file format changes in the future, it will break many people, so it's extremely unlikely to ever change). Alternatively it can be scripted with the following two commands (dotnet nuget doesn't yet have a nuget.exe config equivalent): dotnet new nugetconfig, nuget.exe config -configfile .\nuget.config -set globalPackagesFolder=gpf. Any projects/solutions created in the same directory as this nuget.config, or a subdirectory, will use a subdirectory named gpf where the nuget.config file is, as the global packages directory. Either have your test code/script delete this directory, or use git clean between builds to clear it out.

An alternative is to use SemVer2 preview versions to generate a unique version for each commit/build. This can be done relatively easily using https://github.com/dotnet/Nerdbank.GitVersioning. The issue with this is that your global packages folder will grow with versions of packages that are used once and never again. I'm slowly driving progress on a feature that will help clean up unused packages from the global packages folder. But I prefer the other method of redirecting the global packages folder to a temporary location that gets cleaned up right away.

[cilerler]

@zivkan, I appreciate the detailed explanation. I wasn't aware that we could specify the GlobalNuget folder per solution via nuget.config. It would solve my issue. I appreciate that. Thank you!

[tapika]

I had slightly different issue - but since ticket title suggests the original problem - I'll describe symptoms in here as well.

I have tried to install nuget package from remote using command line like this:

nuget install <package id> -excludeVersion -nocache

It works ok, but it also does not perform install of newer nuget package available on server side.

After cleaning up package folder (overridden by globalPackagesFolder) - nuget would still install older version.

I've added extra call nuget locals http-cache -clear - and after that problem disappeared.

Odd that you need to use extra command line for this, does not work out of box.

I think after maybe 10 minutes it starts to work with newer version as well, but need to wait for that one to happen.

(Nuget version 5.8)

[MrM40]

To force package to only be downloaded to your project-folder and not the cache-folders, place a file called nuget.config in the project-folder with this content:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <config>
    <add key="globalPackagesFolder" value="c:\MyProjectA\libraries\" />
  </config>
</configuration>

Documentation of the nuget.config file: learn.microsoft.com..

Or by setting the environment variable NUGET_PACKAGES, which you can do in the project-file like:

<Project Sdk="Microsoft.NET.Sdk">
	...
	<Target Name="NugetPublicfolder" BeforeTargets="PreBuildEvent">
		<Exec Command="SET NUGET_PACKAGES=C:\MyProjectA\libraries\"/>
	</Target>
	...
</Project>

Notice the nuget source files (.dat) will always first be downloaded to %userprofile%\AppData\Local\NuGet\v3-cache where they will be extracted from, in any case.

[BSarmady]

@zivkan Appreciate your explanation, however, I still have a few issues.

My packages are not in nuget.org, they are in a network location. This location is always available to all our Dev PCs
There are over 80 projects that are using packages from this location.
I noticed a cache folder is created in each Dev PC directly under their user folder (%userprofile%\.nuget)
which in some cases grow over 18GB.

1. Adding a dot . prefix to folder and file names in windows systems doesn't make them hidden, the way it does in linux machines. It only adds nuisance when rename, move or backup the folder.
2. %userprofile% is a common top folder that belongs to user documents and not meant to be used as cache folder and neither %LocalAppData% nor %AppData%. Nuget uses all these 3 locations to store some data.
3. There is a few other Nuget Repo folders created during Visual Studio installarion which are
   • %ProgramFiles(x86)%\NuGet
   • %ProgramFiles(x86)%\Microsoft SDKs\NuGetPackages
   • %ProgramFiles(x86)%\Microsoft SDKs\UWPNuGetPackages
   • %ProgramFiles(x86)%\Microsoft Visual Studio\Shared\NuGetPackages
   • %ProgramFiles(x86)%\Microsoft Visual Studio\Shared\Entity Framework Tools\NuGet Packages

All these make tracking where packages are coming from a lot harder than you think.
Having these many repo location and also a cache location is redundant in our case, since packages are available in our local LAN.
I am also looking for a way to set a system wide package location as Source of Truth so I know for sure that first, packages are not cached in local systems and next, they are always loaded from repo in our Intranet network location.

[zivkan]

Have a look at the docs on nuget.config, specifically the fallback folders section. If you add <clear/> to your solution nuget.config's fallback folder section, then NuGet will only use the global packages folder.

[BSarmady]

@zivkan Thank you for the links. I have read those documents, even mentioned document and Common NuGet configurations explains that nuget stores its data in many locations and Managing the global packages, cache, and temp folders assumes everything is download from internet and should be cached and goes ahead explaining in one paragraph that why it is good to have. (BTW on an off topic issue repositoryPath is a really bad name for environment variable, nuget devs seem to assume developers do not have any other repositories, a better name would be NUGET_REPOSITORY)

These documents still do not explain how can I force nuget not to create many folders all over place, in OS drive (windows C drive) and specifically not to create a cache folder when the repository is actually resides locally in same PC. I'm sure even in Microsoft most dev departments do not have access to internet in their dev machines and use local or Intranet sources.

Adding clear to a configuration file in solution is not feasible, I was asking for a global configuration that applies to every solution that is opened in that dev PC to use same repository by default and not to cache anything. Why nuget can't store its entire data in a single folder like how Gradle does.

I can set 2 environment variable for Gradle that points to a setting folder and executable path, add its executable folder to path and I'm done setting up the Gradle. All of its cache, settings, ... files will go to that folder and no other place.

Here is the setting I use for Gradle

SETX /m PATH "%PATH%;Z:\apps\gradle-8.2.1
SETX /m GRADLE_HOME "Z:\apps\gradle-8.2.1
SETX /m GRADLE_USER_HOME "Z:\settings\gradle

Gradle settings will go to settings folder, cache will be stored in Z:\settings\gradle\cache and I can set apps\gradle folder as readonly too. Note that both Environment variables start with GRADLE_.

This still doesn't stop me from configuring it in project level if I wish to, but gives me ability to keep all my data in my External RAID 1 SSD instead of OS drive (even cache) which means I can pull my drive from this PC and attach to another PC and continue from where I left off without any delay. and since none of these settings are stored in project level, each developer can have their own preferred location (drive name) to keep their data. Mine is Z and my co-worker keeps them in Drive R.