Stop creating a default nuget.config file on first use

[zivkan]

NuGet Product(s) Affected

NuGet.exe, Visual Studio Package Management UI, Visual Studio Package Manager Console, MSBuild.exe, dotnet.exe, NuGet SDK

Current Behavior

Currently, anything using NuGet.Configuration.dll and any of several of the APIs to load settings will create the user profile nuget.config file if it does not already exist. This includes all of NuGet's official tooling (dotnet CLI, VS, msbuild, nuget.exe), but also many other Microsoft and 3rd party apps.

This means that if NuGet wants to change any defaults, whether it's the nuget.org URL, or other settings, such as a proposed auditSource setting, it's very difficult to migrate customers on existing machines to use the new defaults.

Desired Behavior

Customers who only use the default NuGet configuration should not need any nuget.config file anywhere on disk. Unless there is a nuget.config file that uses <clear/> within <packageSource> element, the default nuget.org URL should be used. The automatic/default source should use key nuget.org, since that's what NuGet has been using in the default nuget.config file, so anyone with an existing nuget.config file will have their config overrode the source URL, rather than add a duplicate source.

Decisions will need to be made what should happen when a customer without a nuget.config file does dotnet nuget remove source nuget.org. For example, detect the scenario and output a help message saying to use dotnet new nugetconfig? Automatically create a nuget.config with a <clear/>? nuget.exe, VS, and NuGet Client SDK usage should also be considered, so each of them can provide a suitable experience.

Additional Context

Obviously, any tool using an older version of NuGet.Configuration.dll will continue to create the default nuget.config on first use. But the benefits of implementing this request will come in the future, not immediately on implementation.

Prior art

I'm not aware of any other programming language package manager that requires a config file on disk to work. I have (very, very limited) experience with cargo and npm, and from what I can tell pip also does not create a config file on first use. All of these, as far as I can tell, "hard code" the default package repository URL, and customers needs to create a config file only if they want to change the default configuration.

Linux distribution package managers typically (always?) have configuration files. However, it's common for one version of the package manager to be supported on multiple versions of the OS. Additionally, different package feeds are used to denote which packages are available in each version of the OS. Therefore, in these cases having a configuration file is needed for technical reasons, which NuGet does not have.

Examples of problems this has caused

This proposal attempts to learn from prior problems, hoping to prevent these types of issues happening again in the future.

• migrating from V2 URL to V3 URL

When NuGet version 3 was created, it introduced a newer server API for clients to call over HTTP. However, since most customers had a nuget.config that referenced the V2 URL, this means many customers kept using the V2 URL. In fact, even in late 2023, VS telemetry shows that V2 usage is about 1% of the V3, showing it's difficult for us to get "everyone" to migrate. We even have about 0.5% using both the V2 and V3 nuget.org URL, despite that packages are the same.

• package confusion mitigations issue

One way that prior NuGet client team members attempted to solve the V2 to V3 migration was by detecting when the user-profile nuget.config file is empty, and in this case add nuget.org's v3 URL as a source, unless a second "tracking file" exists. However, the tracking file was not created the default nuget.config file was created.

When the "dependency confusion" blog post was published and became well known, the most common mitigation to secure the supply chain was to remove nuget.org as a package source used directly. Customers were advised to either use a package source such as Azure Artifacts that allows a feed to have multi up-source feeds, which are resolved in a deterministic order, or to mirror nuget.org packages in a feed the customer controls themselves. This led to many customers removing nuget.org as a package source, but then NuGet would immediately re-add nuget.org as a package source, and create the second tracking file. A second removal of nuget.org as a source worked, but it's easy to understand why many customers thought this was a bug.

• Other apps using NuGet.Configuration

Following the implementation of the fix to prevent NuGet from re-adding nuget.org when the tracking file was missing, many customers started experiencing issues with their user-profile not having any sources at all and the NuGet team got many bug reports claiming that NuGet wasn't using nuget.org on clean machines. This is because older NuGet.Configuration.dll versions did exactly this, created the nuget.config without any package sources at all, and two popular apps, PowerShell's PowerShellGet, and Chocolatey, used these versions of NuGet.Configuration.dll. Anyone using either of these apps before using Visual Studio, MSBuild, NuGet.exe, or the .NET SDK for the first time, would end up with a user-profile without nuget.org as a package source. Since Azure Pipelines' Microsoft hosted agents setup the agents images with Chcocolatey, many Azure Pipelines and customers were affected (although Azure Pipelines fixed it quickly by modifying the VM image creation script), in addition to many customers setting up new dev machines.

We worked with Chocolatey and PowerShell so customers using a new Windows ISO should no longer have this issue with either of these products. However, we still occasionally get a bug report from a customer who experiences the issue. Since any app could use NuGet.Configuration, there might be other apps we're not aware of that are still creating these nuget.config files without any package source. Or these other apps might be creating nuget.config files with the v3 package source, which will cause us problems in the future if we ever change the URL again.

[zivkan]

Recently another problem caused by nuget's existing behaviour has become apparent internally:

Some teams are running nuget.exe multiple times in parallel (in this case nuget.exe pack some.proj), before the default user-profile has been created. Although reading the file is guarded by ConcurrencyUtiliy, somehow the processes are failing with "file already in use" errors.