[API Proposal]: Add `Stream` wrappers for memory and text-based types

[jozkee]

Background and motivation

This is meant to bundle #46663, #27156, and #22838.

There's the usual need of wrapping memory and text-based types in a Stream and given that there are multiple implementations spread across different libraries, it would be ideal to have a standardized API for it directly on .NET.

A few examples of the current options available:
CommunityToolkit.HighPerformance provides AsStream() extension methods for the following:
Memory<byte>
ReadOnlyMemory<byte>
IMemoryOwner<byte>
IBufferWriter<byte>

Nerdbank.Streams also provides AsStream() extensions for the following:
IDuplexPipe
PipeReader/PipeWriter
WebSocket
ReadonlySequence<byte>
IBufferWriter<byte>

While I'm not certain of providing support for all the types listed above, I think a good starting point would be the following based on the evidence shown in the issues I bundled.

API Proposal

Originally proposed by @bartonjs on #82801 (comment).

// It will be in a new assembly with a dependency on System.Memory.dll and System.Runtime.dll
namespace System.IO
{
    public partial class StreamFactory
    {
        public static Stream FromText(string text, Encoding encoding);
        public static Stream FromText(Memory<char> text, Encoding encoding);
        public static Stream FromText(ReadOnlyMemory<char> text, Encoding encoding);

        public static Stream FromData(Memory<byte> data);
        public static Stream FromData(ReadOnlyMemory<byte> data);
        public static Stream FromData(ReadOnlySequence<byte> data);
    }
}

API Usage

Example 1:

static HttpClient client = new HttpClient();
static void SendString(string str)
{
    var request = new HttpRequestMessage(HttpMethod.Post, "contoso.com");
    request.Content = new StreamContent(StreamFactory.FromText(str));
    client.Send(request);
}

Example 2:

static unsafe void DoStreamOverSpan(ReadOnlySpan<byte> span)
{
    fixed (byte* ptr = &MemoryMarshal.GetReference(span))
    {
        using (MemoryManager<byte> manager = new PointerMemoryManager<byte>(ptr, span.Length))
        {
            using Stream s = StreamFactory.FromData((ReadOnlyMemory<byte>)manager.Memory);
            // Do something with it...
        }
    }
}

Alternative Design 1

From #82801 (comment).

when two overloads have different semantics the guidelines suggest they shouldn't be overloads, but different names (different method groups).

I had thought this was only about read-only streams. If writing is desired, then perhaps it becomes something more like

public class StreamFactory
{
    public static Stream CreateReadOnly(string text, Encoding encoding);
    public static Stream CreateReadOnly(ReadOnlyMemory<char> text, Encoding encoding);
    public static Stream CreateReadOnly(byte[] data);
    public static Stream CreateReadOnly(ReadOnlyMemory<byte> data);
    public static Stream CreateReadOnly(ReadOnlySequence<byte> data);

    public static Stream CreateWriteOnly(Memory<char> buffer, Encoding encoding);
    public static Stream CreateWriteOnly(byte[] buffer);
    public static Stream CreateWriteOnly(Memory<byte> buffer);

    public static Stream CreateReadWrite(Memory<char> buffer, Encoding encoding);
    public static Stream CreateReadWrite(byte[] buffer);
    public static Stream CreateReadWrite(Memory<byte> buffer);
}

Alternative Design 2

We can think of providing explicit types, e.g: StringStream, instead of simply returning Stream, but I like this way more since the Stream types would be internal and could make it more flexible and maintainable for future improvements e.g: we could start returning a MemoryStream instead of an non-visible one if MemoryStream reaches the point where it finally gets a ReadOnlyMemory<byte> ctor.

Alternative Design 3 (original proposal)

Follows prior-art of libraries listed above, we could provide AsStream() extensions.
Based on comments in this thread pointing out that such extensions don't address a common-enough scenario I've moved this proposal to the alternative section.

namespace System.IO
{
    public static class StreamTextExtensions
    {
        public static Stream AsStream(this ReadOnlyMemory<char> instance, Encoding encoding) { }
        public static Stream AsStream(this string instance, Encoding encoding) { }
    }

    public static class StreamMemoryExtensions
    {
        public static Stream AsStream(this Memory<byte> instance) { }
        public static Stream AsStream(this ReadOnlyMemory<byte> instance) { }
        public static Stream AsStream(this ReadOnlySequence<byte> instance){ }
    }
}

Risks

No response

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

This is meant to bundle #46663, #27156, and #22838.

There's the usual need of wrapping memory and text-based types in a Stream and given that there are multiple implementations spread across different libraries, it would be ideal to have a standardized API for it directly on .NET.

A few examples of the current options available:
CommunityToolkit.HighPerformance provides AsStream() extension methods for the following:
Memory<byte>
ReadOnlyMemory<byte>
IMemoryOwner<byte>
IBufferWriter<byte>

Nerdbank.Streams also provides AsStream() extensions for the following:
IDuplexPipe
PipeReader/PipeWriter
WebSocket
ReadonlySequence<byte>
IBufferWriter<byte>

While I'm not certain of providing support for all the types listed above, I think a good starting point would be the following based on the evidence shown in the issues I bundled.

API Proposal

namespace System.IO
{
    public static class StreamTextExtensions
    {
        public static Stream AsStream(this ReadOnlyMemory<char> instance, Encoding encoding) { }
        public static Stream AsStream(this string instance, Encoding encoding) { }
    }

    public static class StreamMemoryExtensions
    {
        public static Stream AsStream(this Memory<byte> instance) { }
        public static Stream AsStream(this ReadOnlyMemory<byte> instance) { }
        public static Stream AsStream(this ReadOnlySequence<byte> instance){ }
    }
}

API Usage

Example 1:

static HttpClient client = new HttpClient();
static void SendString(string str)
{
    var request = new HttpRequestMessage(HttpMethod.Post, "contoso.com");
    request.Content = new StreamContent(str.AsStream());
    client.Send(request);
}

Example 2:

static unsafe void DoStreamOverSpan(ReadOnlySpan<byte> span)
{
    fixed (byte* ptr = &MemoryMarshal.GetReference(span))
    {
        using (MemoryManager<byte> manager = new PointerMemoryManager<byte>(ptr, span.Length))
        {
            using Stream s = ((ReadOnlyMemory<byte>)manager.Memory).AsStream();
                    
            // Do something with it...
        }
    }
}

Alternative Designs

We can think of providing explicit types instead of simply returning Stream, but I like this way more since the Stream types would be internal and could make it more flexible and maintainable for future improvements e.g: we could start returning a MemoryStream for StreamMemoryExtensions.AsStream(ReadOnlyMemory<byte>) if MemoryStream reaches the point where it finally gets a ReadOnlyMemory<byte> ctor.

Risks

No response

Author:	Jozkee
Assignees:	-
Labels:	api-suggestion, area-System.Memory
Milestone:	-

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

This is meant to bundle #46663, #27156, and #22838.

There's the usual need of wrapping memory and text-based types in a Stream and given that there are multiple implementations spread across different libraries, it would be ideal to have a standardized API for it directly on .NET.

A few examples of the current options available:
CommunityToolkit.HighPerformance provides AsStream() extension methods for the following:
Memory<byte>
ReadOnlyMemory<byte>
IMemoryOwner<byte>
IBufferWriter<byte>

Nerdbank.Streams also provides AsStream() extensions for the following:
IDuplexPipe
PipeReader/PipeWriter
WebSocket
ReadonlySequence<byte>
IBufferWriter<byte>

While I'm not certain of providing support for all the types listed above, I think a good starting point would be the following based on the evidence shown in the issues I bundled.

API Proposal

namespace System.IO
{
    public static class StreamTextExtensions
    {
        public static Stream AsStream(this ReadOnlyMemory<char> instance, Encoding encoding) { }
        public static Stream AsStream(this string instance, Encoding encoding) { }
    }

    public static class StreamMemoryExtensions
    {
        public static Stream AsStream(this Memory<byte> instance) { }
        public static Stream AsStream(this ReadOnlyMemory<byte> instance) { }
        public static Stream AsStream(this ReadOnlySequence<byte> instance){ }
    }
}

API Usage

Example 1:

static HttpClient client = new HttpClient();
static void SendString(string str)
{
    var request = new HttpRequestMessage(HttpMethod.Post, "contoso.com");
    request.Content = new StreamContent(str.AsStream());
    client.Send(request);
}

Example 2:

static unsafe void DoStreamOverSpan(ReadOnlySpan<byte> span)
{
    fixed (byte* ptr = &MemoryMarshal.GetReference(span))
    {
        using (MemoryManager<byte> manager = new PointerMemoryManager<byte>(ptr, span.Length))
        {
            using Stream s = ((ReadOnlyMemory<byte>)manager.Memory).AsStream();
                    
            // Do something with it...
        }
    }
}

Alternative Designs

We can think of providing explicit types instead of simply returning Stream, but I like this way more since the Stream types would be internal and could make it more flexible and maintainable for future improvements e.g: we could start returning a MemoryStream for StreamMemoryExtensions.AsStream(ReadOnlyMemory<byte>) if MemoryStream reaches the point where it finally gets a ReadOnlyMemory<byte> ctor.

Risks

No response

Author:	Jozkee
Assignees:	-
Labels:	api-suggestion, area-System.IO, untriaged
Milestone:	-

[jozkee]

cc @AArnott @Sergio0694 @stephentoub.

[bartonjs]

I don't think that making a stream from a string (or byte buffer) is common enough to warrant an extension method. I'd rather see it as static non-extension methods on Stream, such as

public partial class Stream
{
    public static Stream FromText(string text, Encoding encoding);
    public static Stream FromText(ReadOnlyMemory<char> text, Encoding encoding);

    public static Stream FromData(byte[] data);
    public static Stream FromData(ReadOnlyMemory<byte> data);
    public static Stream FromData(ReadOnlySequence<byte> data);
}

[jozkee]

I don't know how common is too common but the top StackOverflow result has 1.2k upvotes in the top answer and is quite suboptimal.

public static Stream GenerateStreamFromString(string s)
{
    var stream = new MemoryStream();
    var writer = new StreamWriter(stream);
    writer.Write(s);
    writer.Flush();
    stream.Position = 0;
    return stream;
}

[AArnott]

When it comes to converting text to a Stream, I don't know that your anti-sample is all that bad. You'll have to encode the text either way. I suppose you could encode it in a streaming fashion instead of all up front, but that would add a great deal of complexity, and for a string (which shouldn't be too large in the common case) the complexity may make it slower.

[jozkee]

@AArnott that's fair. It's possible that encoding on the go is not more optimal in all scenarios, especially on small strings as you say.