Add ability to perform zero byte reads in StreamPipeReader

[davidfowl]

Background and Motivation

Bufferless reads are a technique used in a couple of places in ASP.NET Core to avoid renting memory from the memory pool if there's no data available on the socket; this helps when trying to reduce the memory usage for idle connections for applications that are trying to handle a large number of concurrent connections (like websocket based applications).

We'd like to use this for websocket applications to remove the need to allocate any memory (see #37122 for an example)

Proposed API

namespace System.Collections.Generic
{
    public class StreamPipeReaderOptions 
    {
+      public bool WaitForData { get; set; }
    }

WaitForData tells the PipeReader implementation that it should wait for data from the underlying stream before allocating memory. It relies on the stream supporting zero byte reads (which networking and SSL do)

Usage Examples

var reader = PipeReader.Create(stream, new StreamPipeReaderOptions { WaitForData = true })

Numbers in SSLStream

• During TLS Handshakes we allocate an 8K buffer while waiting for the handshake
• During pending TLS reads we allocate a 16K buffer per read
  • The caller then allocates a 4K buffer on top of this so we burn 20K per connection in pooled buffers (that shouldn't be pooled, see SSLStream should not use the ArrayPool for it's internal buffers #49001)

[terrajobst]

Video

• Zero byte reads are technically a violation of the Stream.Read() contract, but granted many streams probably support it by accident.
• We should have an API on Stream instead
• We should validate the SslStream, SocketListenerStream, NetworkStream, and other compression streams
• This should be investigated and the API proposal should be updated

namespace System.IO.Pipelines
{
    public class StreamPipeReaderOptions
    {
        public bool WaitForData { get; set; }
    }
}
namespace System.IO
{
    public partial class Stream
    {
        public virtual Task WaitForDataAsync() => return Task.CompletedTask;
    }
}