Slow file access times when accessing NFS server

[DomiStyle]

Description

NFS performance is awful in .NET while Python and Bash have no problem accessing files at expected speeds.

As the tests below show, .NET needs around 6 seconds to access a file on an NFS share, doesn't matter if reading or writing. File size doesn't matter, access times are slow. Creating folders, deleting folders and deleting files works normally.

Configuration

.NET 5.0.103
Ubuntu 20.04.2 LTS x64 (with 5.11 kernel)
Issue happens on all my .NET applications across various Linux distros

Regression?

Didn't work since at least the introduction of .NET Core 3.0

Other information

Test files are available in this repository.

All times in milliseconds

Network speed according to iperf3: 9.13 Gbits/sec
Files on NFS server are on HDD

1GB file size

Shell:

From local to NFS: 10847
From NFS to local: 636

.NET:

From local to NFS: 16814
From NFS to local: 6776
Time to create folder: 3
Time to delete folder: 0
Time to create 10MB file: 6252
Time to read 10MB file: 6235
Time to delete 10MB file: 1

Python:

From local to NFS:  10812.782049179077
From NFS to local:  571.5739727020264

10MB file size

Shell:

From local to NFS: 167
From NFS to local: 8

.NET:

From local to NFS: 6352
From NFS to local: 6208
Time to create folder: 4
Time to delete folder: 0
Time to create 10MB file: 6319
Time to read 10MB file: 6216
Time to delete 10MB file: 1

Python:

From local to NFS:  183.42185020446777
From NFS to local:  6.532192230224609

10MB file size
Local control test (SSD)

Shell:

From local to local: 1
From local to local: 1

.NET:

From local to local: 4
From local to local: 0
Time to create folder: 6
Time to delete folder: 0
Time to create 10MB file: 5
Time to read 10MB file: 3
Time to delete 10MB file: 0

Python:

From local to local:  0.05602836608886719
From local to local:  0.010967254638671875

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[DomiStyle]

Area label is probably area-System.IO.

@jeffhandley @adamsitnik @carlossanlop @jozkee

[jeffhandley]

Removing "needs further triage" since we've marked this as "up-for-grabs".

[DomiStyle]

I can still reproduce this on .NET 6. I tested a few different server distros to see if there are differences:

• Ubuntu 20.04.3: Same behavior as in original issue
• Debian 11: Same behavior as in original issue
• Alpine Linux 3.15: Only the very first read after a NFS server restart is (very) delayed, all successive requests work as expected
• Fedora Server 35: Same behavior as Alpine Linux

The control applications in Python and Bash still experience none of these issues. The client is always Ubuntu 21.10.

If the Python or Bash control application accesses the NFS mount on Alpine or Fedora first the .NET application will have no further issues afterwards.