Microsoft.Extensions.FileProviders.Physical.Tests disabled on Unix systems

[tarekgh]

https://github.com/dotnet/runtime/pull/56183/checks?check_run_id=3138813912
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-56183-merge-f78074ea8e6047b2a6/Microsoft.Extensions.FileProviders.Physical.Tests/console.ac3f3b03.log?sv=2019-07-07&se=2021-08-11T22%3A22%3A03Z&sr=c&sp=rl&sig=ywBXxPkq8EuFYEZl2jIf5dvaPIqJDBOgm1Phou2QmU0%3D

Console log: 'Microsoft.Extensions.FileProviders.Physical.Tests' from job f78074ea-8e60-47b2-a6e4-961f30f8afa1 workitem 7254ea10-54b1-48a3-9672-5fbd419dfa26 (osx.1013.amd64.open) executed on machine dci-mac-build-083.local
+ ./RunTests.sh --runtime-path /tmp/helix/working/AB72097D/p
----- start Thu Jul 22 15:24:05 PDT 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/AB72097D/p/dotnet exec --runtimeconfig Microsoft.Extensions.FileProviders.Physical.Tests.runtimeconfig.json --depsfile Microsoft.Extensions.FileProviders.Physical.Tests.deps.json xunit.console.dll Microsoft.Extensions.FileProviders.Physical.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/private/tmp/helix/working/AB72097D/w/A0CB0914/e /private/tmp/helix/working/AB72097D/w/A0CB0914/e
  Discovering: Microsoft.Extensions.FileProviders.Physical.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.FileProviders.Physical.Tests (found 76 of 92 test cases)
  Starting:    Microsoft.Extensions.FileProviders.Physical.Tests (parallel test collections = on, max threads = 4)
Unhandled exception. System.IO.FileNotFoundException: Could not find file '/var/folders/vn/knn7pvpd2y12gn_qhnnzqps80000gy/T/nnep5noo.kpg/e4a4s2kc.z55'.
File name: '/var/folders/vn/knn7pvpd2y12gn_qhnnzqps80000gy/T/nnep5noo.kpg/e4a4s2kc.z55'
   at System.IO.FileSystem.ResolveLinkTarget(String linkPath, Boolean returnFinalTarget, Boolean isDirectory) in /_/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Unix.cs:line 603
   at Microsoft.Extensions.FileProviders.Physical.FileSystemInfoHelper.GetFileLinkTargetLastWriteTimeUtc(FileInfo fileInfo) in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/Internal/FileSystemInfoHelper.cs:line 54
   at Microsoft.Extensions.FileProviders.Physical.PollingFileChangeToken.GetLastWriteTimeUtc() in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PollingFileChangeToken.cs:line 57
   at Microsoft.Extensions.FileProviders.Physical.PollingFileChangeToken.get_HasChanged() in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PollingFileChangeToken.cs:line 101
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.RaiseChangeEvents(Object state) in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PhysicalFilesWatcher.cs:line 437
   at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Timer.cs:line 673
   at System.Threading.TimerQueue.FireNextTimers() in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Timer.cs:line 326

Tagging subscribers to this area: @maryamariyan, @dotnet/area-extensions-filesystem
See info in area-owners.md if you want to be subscribed.

Issue Details

---

https://github.com/dotnet/runtime/pull/56183/checks?check_run_id=3138813912
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-56183-merge-f78074ea8e6047b2a6/Microsoft.Extensions.FileProviders.Physical.Tests/console.ac3f3b03.log?sv=2019-07-07&se=2021-08-11T22%3A22%3A03Z&sr=c&sp=rl&sig=ywBXxPkq8EuFYEZl2jIf5dvaPIqJDBOgm1Phou2QmU0%3D

Console log: 'Microsoft.Extensions.FileProviders.Physical.Tests' from job f78074ea-8e60-47b2-a6e4-961f30f8afa1 workitem 7254ea10-54b1-48a3-9672-5fbd419dfa26 (osx.1013.amd64.open) executed on machine dci-mac-build-083.local
+ ./RunTests.sh --runtime-path /tmp/helix/working/AB72097D/p
----- start Thu Jul 22 15:24:05 PDT 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/AB72097D/p/dotnet exec --runtimeconfig Microsoft.Extensions.FileProviders.Physical.Tests.runtimeconfig.json --depsfile Microsoft.Extensions.FileProviders.Physical.Tests.deps.json xunit.console.dll Microsoft.Extensions.FileProviders.Physical.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/private/tmp/helix/working/AB72097D/w/A0CB0914/e /private/tmp/helix/working/AB72097D/w/A0CB0914/e
  Discovering: Microsoft.Extensions.FileProviders.Physical.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.FileProviders.Physical.Tests (found 76 of 92 test cases)
  Starting:    Microsoft.Extensions.FileProviders.Physical.Tests (parallel test collections = on, max threads = 4)
Unhandled exception. System.IO.FileNotFoundException: Could not find file '/var/folders/vn/knn7pvpd2y12gn_qhnnzqps80000gy/T/nnep5noo.kpg/e4a4s2kc.z55'.
File name: '/var/folders/vn/knn7pvpd2y12gn_qhnnzqps80000gy/T/nnep5noo.kpg/e4a4s2kc.z55'
   at System.IO.FileSystem.ResolveLinkTarget(String linkPath, Boolean returnFinalTarget, Boolean isDirectory) in /_/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Unix.cs:line 603
   at Microsoft.Extensions.FileProviders.Physical.FileSystemInfoHelper.GetFileLinkTargetLastWriteTimeUtc(FileInfo fileInfo) in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/Internal/FileSystemInfoHelper.cs:line 54
   at Microsoft.Extensions.FileProviders.Physical.PollingFileChangeToken.GetLastWriteTimeUtc() in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PollingFileChangeToken.cs:line 57
   at Microsoft.Extensions.FileProviders.Physical.PollingFileChangeToken.get_HasChanged() in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PollingFileChangeToken.cs:line 101
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.RaiseChangeEvents(Object state) in /_/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PhysicalFilesWatcher.cs:line 437
   at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Timer.cs:line 673
   at System.Threading.TimerQueue.FireNextTimers() in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Timer.cs:line 326

Author:	tarekgh
Assignees:	-
Labels:	area-Extensions-FileSystem, untriaged
Milestone:	-

[jozkee]

I think this is because retrieval of the link is not atomic and happens when the file no longer exists between the call to LinkTarget and ResolveLinkTarget.

runtime/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/Internal/FileSystemInfoHelper.cs

Lines 52 to 57 in 47e82c1

	if (fileInfo.LinkTarget != null)
	{
	FileSystemInfo targetInfo = fileInfo.ResolveLinkTarget(returnFinalTarget: true);
	if (targetInfo.Exists)
	{
	return targetInfo.LastWriteTimeUtc;

A couple of options:

• Use fix proposed by @ericstj Make polling use the symbolic link target's LastWriteTime #55664 (comment) instead of the current fix.
• Wrap this logic in a try-catch so we ensure that querying the target is only a best-effort.
• If we wanted to solve this with public APIs, I think we would need something like FileInfo/DirectoryInfo: enable transparent handling of symbolic links #52908 which gets the target information in one go.
• Or add a TryResolveLinkTarget API that doesn't throw on IO errors.

cc @adamsitnik @carlossanlop @jeffhandley

[ericstj]

To be clear, my suggestion also involves public APIs for getting the modified time on the target of link. The prototype i shared did it internally which should be avoided if we’re making IO changes. If we need it customers should get it too.

[hoyosjs]

This started in the last two days: https://runfo.azurewebsites.net/search/tests/?q=started%3A%7E7+definition%3Aruntime+name%3A%22Microsoft.Extensions.FileProviders.Physical.Tests%22 Somehow it's able to crash XUnit. @jozkee is it possible to disable anything that will unblock this? It's quickly becoming a heavy hitter.

[jozkee]

is it possible to disable anything that will unblock this?

Yes.

Seems that all the errors are on OSX and Linux, can we just disable the tests for those platforms? I still would like to see what issues could occur in windows since local repro is hard.