File.ReadAllBytes* should support non-seekable files

[adamsitnik]

As of today, File.WriteAllBytes* APIs support non-seekable files, but File.ReadAllBytes* don't.

The following call to FileStream.Length is going to throw:

runtime/src/libraries/System.Private.CoreLib/src/System/IO/File.cs

Line 336 in d033e1a

long fileLength = fs.Length;

The funny thing is that we already almost support it:

runtime/src/libraries/System.Private.CoreLib/src/System/IO/File.cs

Lines 341 to 348 in d033e1a

	else if (fileLength == 0)
	{
	#if !MS_IO_REDIST
	// Some file systems (e.g. procfs on Linux) return 0 for length even when there's content.
	// Thus we need to assume 0 doesn't mean empty.
	return ReadAllBytesUnknownLength(fs);
	#endif
	}

We need:

• test that shows the broken scenario
• a fix

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

As of today, File.WriteAllBytes* APIs support non-seekable files, but File.ReadAllBytes* don't.

The following call to FileStream.Length is going to throw:

runtime/src/libraries/System.Private.CoreLib/src/System/IO/File.cs

Line 336 in d033e1a

long fileLength = fs.Length;

The funny thing is that we already almost support it:

runtime/src/libraries/System.Private.CoreLib/src/System/IO/File.cs

Lines 341 to 348 in d033e1a

	else if (fileLength == 0)
	{
	#if !MS_IO_REDIST
	// Some file systems (e.g. procfs on Linux) return 0 for length even when there's content.
	// Thus we need to assume 0 doesn't mean empty.
	return ReadAllBytesUnknownLength(fs);
	#endif
	}

We need:

• test that shows the broken scenario
• a fix

Author:	adamsitnik
Assignees:	-
Labels:	area-System.IO, up-for-grabs
Milestone:	7.0.0

[lateapexearlyspeed]

Hi @adamsitnik could you please help on question:
how to create non-seekable readable FileStream in Windows by calling path based new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1, FileOptions.SequentialScan) which is used in File.ReadAllBytes() ?
I can only create non-seekable FileStream by creating namedPipe then new FileStream(new SafeFileHandle(namedPipeServer.SafePipeHandle.DangerousGetHandle(), true), FileAccess.Read) which is code from test

[adamsitnik]

@lateapexearlyspeed Appologies for the delay, I've missed GH notificaiton for your question.

You can take a look at the tests that I am trying to add in #58506 :

runtime/src/libraries/System.IO.FileSystem/tests/RandomAccess/NonSeekable_AsyncHandles.cs

Lines 21 to 25 in 2b1510f

	string pipeName = FileSystemTest.GetNamedPipeServerStreamName();
	string pipePath = Path.GetFullPath($@"\\.\pipe\{pipeName}");
	using (var server = new NamedPipeServerStream(pipeName, PipeDirection.InOut))
	using (SafeFileHandle clientHandle = File.OpenHandle(pipePath, FileMode.Open, access, FileShare.None, FileOptions.Asynchronous))

in your case you should be able to create a FileStream from given path. Sth like this:

string pipeName = FileSystemTest.GetNamedPipeServerStreamName();
string pipePath = Path.GetFullPath($@"\\.\pipe\{pipeName}");

using (var server = new NamedPipeServerStream(pipeName, PipeDirection.InOut))
using (FileStream fs = new FileStream(pipePath, FileMode.Open, access, FileShare.None, FileOptions.Asynchronous))

[lateapexearlyspeed]

@adamsitnik Thanks for help, test cases finished in PR, sync version test for File.ReadAllBytes() passed but async version test for File.ReadAllBytesAsync() failed, with exception:

System.IO.IOException : The parameter is incorrect. : '.\pipe\a0b5cd9498c54bb4a6ea30fd57edf8d8'
at System.IO.File.InternalReadAllBytesUnknownLengthAsync(FileStream fs, CancellationToken cancellationToken) in //src/libraries/System.Private.CoreLib/src/System/IO/File.cs:line 812
at System.IO.Tests.File_ReadWriteAllBytesAsync.ReadAllBytesAsync_NonSeekableFileStream() in //src/libraries/System.IO.FileSystem/tests/File/ReadWriteAllBytesAsync.cs:line 202
--- End of stack trace from previous location ---

which can be reproduced by following call:

            string pipeName = FileSystemTest.GetNamedPipeServerStreamName();
            string pipePath = Path.GetFullPath($@"\\.\pipe\{pipeName}");

            var namedPipeWriterStream = new NamedPipeServerStream(pipeName, PipeDirection.Out);
            var fs = new FileStream(pipePath, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1, FileOptions.Asynchronous | FileOptions.SequentialScan);
            await fs.ReadAsync(new byte[3].AsMemory()); // Here will throw same exception as test case throws

Not sure is there issue inside FileStream for async operation ?

[adamsitnik]

Not sure is there issue inside FileStream for async operation ?

Most likely. I am going to build your fork and try to repro it locally

[adamsitnik]

It turned out that the official docs were not telling the exact truth.

For an hFile that does not support byte offsets, Offset and OffsetHigh are ignored.

I've added an handle.CanSeek condition to your PR (#58434) and the tests are passing now.

cc @stephentoub