API Proposal: Span<char> from null-terminated char*

[fbrosseau]

Background and Motivation

PInvoke scenarios (mostly those on Windows) interact a lot with null-terminated wide strings.

String has always had a ctor(char*), which in .Net Core now uses a highly optimized wcslen, internal to the framework.

I am proposing the equivalent functionality, minus the allocation+copy, using Span<>.
Given a null-terminated wide string as input (in the shape of unsafe char*), return a Span whose length is the count of characters before the null.

I am proposing Span<> and not ReadOnlySpan<>, to let the caller decide what to do with the result, according to their specific scenario and the nature of their char-pointer.
As Jan mentionned, the big majority of usecases are const, so the API should be ReadOnlySpan.

I am not proposing to also add the equivalent narrow ctor(sbyte*) version for span, as this one cannot be implemented without allocating a new buffer.
UTF8 (byte*) returns ReadOnlySpan<byte>.

Implementing the same functionality with just public API is possible, but very awkward and undocumented internal behavior (this is more or less what wcslen does, with error-checking omitted):

int len = MemoryExtensions.IndexOf(new ReadOnlySpan<char>(value, int.MaxValue), '\0');
return new Span<char>(value, len);

Proposed API

namespace System.Runtime.InteropServices
{
    public static class MemoryMarshal
    {
+       public static unsafe ReadOnlySpan<char> CreateFromNullTerminated(char* value);
+       public static unsafe ReadOnlySpan<byte> CreateFromNullTerminated(byte* value);
    }
}

The implementation of this proposal is also trivial with the existing tools in the framework (null is non-throwing, the behavior of string ctor(char*)):

public static unsafe ReadOnlySpan<char> CreateFromNullTerminated(char* value)
{
    if (value == null)
        return default;
 
    int count = string.wcslen(value);
    if (count == 0)
        return default;

    return new ReadOnlySpan<char>(value, count);
}

Usage Examples

['ptr' is a char* that points to null-terminated string "abc\0"]

var span = MemoryMarshal.CreateFromNullTerminated(ptr);

['span' is now a Span<char> of length 3.]

Alternative Designs

Risks

[svick]

Is it common that the total buffer size is unknown? I worry about the situation where the null terminator is missing, which would mean the created Span points to random regions of memory. Would it make more sense to add a parameter for the buffer size?

public static unsafe Span<char> CreateSpanFromNullTerminatedString(char* value, int maxLength);

If \0 is not found within maxLength characters, an exception is thrown.

If you actually do not know the size of the buffer, you can specify int.MaxValue as maxLength.

[fbrosseau]

A missing null terminator in a null-terminated string would be a breach of a very fundamental contract that is present in 99% of c-style APIs, including big parts of the Win32 API. Null-terminated strings of unknown length are everywhere.

A null-terminated string without a null is a bug in the code that generated it, and I don't think the framework should patch around that in what is unsafe code in the first place (because of the use of pointers).

[jkotas]

Real-world example of this pattern:

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/CalendarData.Icu.cs

Line 436 in ed6eda5

var calendarStringSpan = new ReadOnlySpan<char>(calendarStringPtr, string.wcslen(calendarStringPtr));

[jkotas]

• Should the API return ReadOnlySpan<char>? The null-terminated strings are typically meant to be read-only.
• The prosed name is a mouthful. The existing interop APIs that work with null-terminated string (e.g. new string(char*) or Marshal.PtrToStringUni) do not mention zero-terminated in the name. What about MemoryMarshal.CreateSpanFromString(char*) or even just MemoryMarshal.CreateSpan(char*) ?

[GrabYourPitchforks]

int len = MemoryExtensions.IndexOf(new ReadOnlySpan<char>(value, int.MaxValue), '\0');

FWIW, I would not recommend this in a production application. This takes advantage of undocumented behavior within the framework, and this behavior is subject to change.

If applications need an immediate workaround, I'd suggest ReadOnlySpan<char> span = new string(charPtr).AsSpan();. Yes, it allocates, but it's 100% guaranteed to work and uses only supported, documented APIs. Or write the for loop manually if you need to avoid allocations.

For a first-class API, I like the name MemoryMarshal.CreateSpan(char*).

[GrabYourPitchforks]

This takes advantage of undocumented behavior within the framework, and this behavior is subject to change.

I should probably add a comment in the source clarifying this.

[fbrosseau]

This takes advantage of undocumented behavior within the framework, and this behavior is subject to change.

This is why I am making this public API proposal :).

And I agree with any name suggestion, CreateSpan sounds great, I knew my name suggestion was not good.

[jkotas]

I have updated the proposal with the shorter name

[fbrosseau]

Thanks