[API Proposal]: Base64.IsValid

[stephentoub]

Background and motivation

The Base64 class provides efficient methods for encoding and decoding base64 data, but it doesn't provide any means for validating Base64-encoded data is properly encoded, at least not without having output memory into which to decode the resulting data. For scenarios that, for example, want to validate configuration data promptly and that might need the results of that configuration ever or until later, it's desirable to support an efficient means for validating Base64-encoded data without requiring the output memory, which then also enables the decoding/validation to be performed faster.

API Proposal

namespace System.Buffers.Text;

public static class Base64
{
+    public static bool IsValid(ReadOnlySpan<char> base64Text);
+    public static bool IsValid(ReadOnlySpan<byte> base64Text);
}

API Usage

string base64Text = ...;
if (!Base64.IsValid(base64Text))
    throw new InvalidConfigurationException(...);

Alternative Designs

• We could expose such methods on the new Ascii class instead, since base64-encoded data is all ASCII. However, we already have two different places for Base64 functionality (Convert and Base64), and ideally we'd use the Base64 class as the central location moving forward for Base64-related functionality (we might even want to duplicate the Convert APIs for working with chars onto the Base64 type for discoverability, or potentially create streaming char-based versions to parallel the streaming byte-based versions already there).
• The existing Encode/Decode methods call the source text parameter "utf8"; we could do the same here, but only for the byte-based overload, which is then inconsistent between the overloads.

Risks

No response

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

The Base64 class provides efficient methods for encoding and decoding base64 data, but it doesn't provide any means for validating Base64-encoded data is properly encoded, at least not without having output memory into which to decode the resulting data. For scenarios that, for example, want to validate configuration data promptly and that might need the results of that configuration ever or until later, it's desirable to support an efficient means for validating Base64-encoded data without requiring the output memory, which then also enables the decoding/validation to be performed faster.

API Proposal

namespace System.Buffers.Text;

public static class Base64
{
+    public static bool IsValid(ReadOnlySpan<char> base64Text);
+    public static bool IsValid(ReadOnlySpan<byte> base64Text);
}

API Usage

string base64Text = ...;
if (!Base64.IsValid(base64Text))
    throw new InvalidConfigurationException(...);

Alternative Designs

• We could expose such methods on the new Ascii class instead, since base64-encoded data is all ASCII. However, we already have two different places for Base64 functionality (Convert and Base64), and ideally we'd use the Base64 class as the central location moving forward for Base64-related functionality (we might even want to duplicate the Convert APIs for working with chars onto the Base64 type for discoverability, or potentially create streaming char-based versions to parallel the streaming byte-based versions already there).

Risks

No response

Author:	stephentoub
Assignees:	-
Labels:	api-suggestion, area-System.Memory
Milestone:	8.0.0

[terrajobst]

Video

• Looks good as proposed
  • We should consider answering the question whether Decode will work, rather than just checking whether it's syntactically valid Base64, which would be more useful.

namespace System.Buffers.Text;

public static class Base64
{
    public static bool IsValid(ReadOnlySpan<char> base64Text);
    public static bool IsValid(ReadOnlySpan<char> base64Text, out int decodedLength);
    public static bool IsValid(ReadOnlySpan<byte> base64Text);
    public static bool IsValid(ReadOnlySpan<byte> base64Text, out int decodedLength);
}

[GrabYourPitchforks]

Here are some observations from quick experimentation, as discussed during review.

Both Convert.FromBase64String and Base64.Decode handle non-canonical forms (RFC 4648, Sec. 3.5) equivalently. Padding bits are discarded without being checked to see if they're all-zero.

Convert.FromBase64String allows whitespace, while Base64.Decode forbids whitespace. (See RFC 4648, Sec. 3.1.)

[stephentoub]

Convert.FromBase64String allows whitespace, while Base64.Decode forbids whitespace.

That's an unfortunate discrepancy. The cleanest option from my perspective would be to update Base64.Decode* to allow whitespace, and then Base64.IsValid would similarly ignore them.

@GrabYourPitchforks, what do you think we should do here? Base64.IsValid aside, it's strange to me that we have these two different base64-decoding routines that treat whitespace (in particular newlines) differently.

[gfoidl]

These APIs could be implemented based on #68328 (comment)

namespace System.Buffers.Text;

public static class Base64
{
    public static bool IsValid(ReadOnlySpan<char> base64Text) => base64Text.IndexOfAnyExcept(/* base64 alphabet + allowed whitespace */) < 0;
    public static bool IsValid(ReadOnlySpan<byte> base64Text) => => base64Text.IndexOfAnyExcept(/* base64 alphabet + allowed whitespace */) < 0;

    public static bool IsValid(ReadOnlySpan<char> base64Text, out int decodedLength)
    {
        int indexOfFirstNonBase64 = base64Text.IndexOfAnyExcept(/* base64 alphabet + allowed whitespace */);

        if (indexOfFirstNonBase64 < 0)
        {
            decodedLength = base64Text.Length;    // probably wrong, see comment below
            return true;
        }

        decodedLength = indexOfFirstNonBase64 & -4;
        return false;
    }

    public static bool IsValid(ReadOnlySpan<byte> base64Text, out int decodedLength)
    {
        // the same
    }
}

Here I assume that decodedLength in the case of being not valid returns length that could actually be decoded (i.e. tha "valid part").

[stephentoub]

These APIs could be implemented based

That's the intent :-) There are a bunch of places the new api will be useful.

[stephentoub]

Here I assume that decodedLength in the case of being not valid returns length that could actually be decoded

We didn't talk about what it would be in the invalid case, but your suggestion is reasonable. I think the intent, however, was for it to be the required output buffer size for the decoded data rather than the input length; that's particularly relevant if whitespace is ignored.

[gfoidl]

was for it to be the required output buffer size for the decoded data

So for documentation that would mean:
IsValid = true -> required buffer size that can hold the decoded data
IsValid = false -> up until this length data can be decoded, then there's non base64 (useful for failure message, etc.)