Restore and HTTP timeout (probably due to TCP parallel connections being throttled)

[xen2]

Please read the following information before posting the issue.

Details about Problem

NuGet product used: NuGet client API

NuGet version (x.x.x.xxx): 5.0

OS version: win10

Detailed repro steps so we can see the same problem

Only happen on slow connections:

1. Install Xenko Launcher (https://xenko.com/download)

2. Install latest version of Xenko
   Note: Xenko is quite big, around 60 packages with a total size of 300 MB.

3. Some people (with slow connections) have timeout issues:
   Can't install Xenko stride3d/stride#310

Other suggested things

My best guess on what's happening:
Some ISP throttle TCP connections: even though NuGet open 10 TCP connections, only 2 will actively progress and the other will be put on hold until some other finishes. Given a slow enough download rate, 2 running download could last more than 100 second and will result in other pending connections to be considered timeout by NuGet (note: the timeout is not at HTTP level but just NuGet measuring time of a call and cancelling if too long).

Note: timeout also happen on very simple/small files that should not timeout and are perfectly working, such as https://api.nuget.org/v3-flatcontainer/xenko.assets/index.json (source: stride3d/stride#310)

Unsuccessful attempts to mitigate:

• I could find RestoreRequest.MaxDegreeOfConcurrency but I think it only affect number of high level tasks, not the number of parallel downloads.
• Also, I couldn't find a way to set a better timeout through API: HttpSourceCachedRequest.RequestTimeout is assigned (indirectly) from HttpSourceRequest.DefaultRequestTimeout which is a static field.

Verbose Logs

Recently got a full NuGet log of someone having the issue:
XenkoLauncherLog.txt

Sample Project

Our NuGet package download high-level function is there:
https://github.com/xenko3d/xenko/blob/master/sources/assets/Xenko.Core.Packages/NugetStore.cs#L293

[rrelyea]

It seems like your users have been hitting this for a while. What different versions of NuGet are they hitting that with? How are you calling NuGet restore?

[xen2]

@rrelyea With 4.x and 5.x.
We are calling NuGet directly through API (cf first post, link to source).
I suspect a nuget install Xenko.GameStudio -Version 3.1.0.1-beta02-0550 might result in the same error but I don't have the issue with my ISP/connection so can't confirm.

[xen2]

@rrelyea I would like to submit a PR to make HttpSourceRequest.DefaultRequestTimeout customizable through API.
Is it OK to remove the readonly modifier, or would you like it to be passed as a parameter in all the places creating HttpSourceRequest or HttpSourceCachedRequest?

[mrbaley]

This can also happen when a ([very slow] corporate) firewall scans the packages and needs more than 100000ms for that.