[API Proposal]: MemoryExtensions.ContainsAny{Except}

[MihaZupan]

Background and motivation

Similar to how we have span.IndexOf(T) and span.Contains(T), I'm proposing a contains variant of IndexOfAny{Except}.
These methods would be semantically equivalent to IndexOfAny{Except} >= 0.

The motivation behind the API is similar to the original Contains (#25228, #33785 (comment)):

• we can avoid some costs if we know that the caller doesn't care about the exact position of a match
• ContainsAny/!ContainsAny is more readable than various combinations of IndexOfAny + >= 0 < 0 != -1 ...

Example performance numbers for IndexOfAny vs ContainsAny for a set of Ascii values

Method	Length	MatchAtStart	Mean
IndexOfAny	1	False	1.642 ns
ContainsAny	1	False	1.206 ns
IndexOfAny	1	True	1.828 ns
ContainsAny	1	True	1.341 ns
IndexOfAny	7	False	7.455 ns
ContainsAny	7	False	5.057 ns
IndexOfAny	7	True	1.873 ns
ContainsAny	7	True	1.377 ns
IndexOfAny	8	False	2.207 ns
ContainsAny	8	False	2.064 ns
IndexOfAny	8	True	3.267 ns
ContainsAny	8	True	2.459 ns
IndexOfAny	16	False	2.189 ns
ContainsAny	16	False	2.033 ns
IndexOfAny	16	True	3.256 ns
ContainsAny	16	True	2.465 ns
IndexOfAny	32	False	2.908 ns
ContainsAny	32	False	2.613 ns
IndexOfAny	32	True	3.909 ns
ContainsAny	32	True	2.283 ns
IndexOfAny	64	False	3.445 ns
ContainsAny	64	False	3.155 ns
IndexOfAny	64	True	3.261 ns
ContainsAny	64	True	1.942 ns

API Proposal

namespace System;

public static class MemoryExtensions
{
    // Existing
    public static int IndexOf<T>(this ReadOnlySpan<T> span, T value) where T : IEquatable<T>?;
    public static bool Contains<T>(this ReadOnlySpan<T> span, T value) where T : IEquatable<T>?;
    public static int IndexOfAny<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static int IndexOfAnyExcept<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static int IndexOfAny(this ReadOnlySpan<char> span, SearchValues<string> values);

    // Proposed
    public static bool ContainsAny<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAny(this ReadOnlySpan<char> span, SearchValues<string> values);
}

API Usage

-if (text.IndexOfAny(s_invalidChars) >= 0)
+if (text.ContainsAny(s_invalidChars))

-if (text.IndexOfAnyExcept(s_allowedChars) >= 0)
+if (text.ContainsAnyExcept(s_allowedChars))

Alternative Designs

IndexOfAny has many overloads and variants. Should any of them also have ContainsAny variants?

static bool ContainsAnyExcept(this ReadOnlySpan<T> span, T value);
static bool ContainsAny{Except}(this ReadOnlySpan<T> span, T value0, T value1);
static bool ContainsAny{Except}(this ReadOnlySpan<T> span, T value0, T value1, value2);
static bool ContainsAny{Except}(this ReadOnlySpan<T> span, ReadOnlySpan<T> values);
static bool ContainsAny{Except}InRange(this ReadOnlySpan<T> span, T lowInclusive, T highInclusive);
static bool ContainsAny{Except}WhiteSpace(this ReadOnlySpan<T> span);

static bool ContainsNewLine(ReadOnlySpan<char> value) =>
-   value.IndexOfAny('\r', '\n') >= 0;
+   value.ContainsAny('\r', '\n');

Risks

More methods to think about - ContainsAny/ContainsAnyExcept would be a new set of overloads on a common type.

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Similar to how we have span.IndexOf(T) and span.Contains(T), I'm proposing a contains variant of IndexOfAny{Except}.
These methods would be semantically equivalent to IndexOfAny{Except} >= 0.

The motivation behind the API is similar to the original Contains (#25228, #33785 (comment)):

• we can avoid some costs if we know that the caller doesn't care about the exact position of a match
• ContainsAny/!ContainsAny is more readable than various combinations of IndexOfAny + >= 0 < 0 != -1 ...

Example performance numbers for IndexOfAny vs ContainsAny for a set of Ascii values

Method	Length	MatchAtStart	Mean
IndexOfAny	1	False	1.642 ns
ContainsAny	1	False	1.206 ns
IndexOfAny	1	True	1.828 ns
ContainsAny	1	True	1.341 ns
IndexOfAny	7	False	7.455 ns
ContainsAny	7	False	5.057 ns
IndexOfAny	7	True	1.873 ns
ContainsAny	7	True	1.377 ns
IndexOfAny	8	False	2.207 ns
ContainsAny	8	False	2.064 ns
IndexOfAny	8	True	3.267 ns
ContainsAny	8	True	2.459 ns
IndexOfAny	16	False	2.189 ns
ContainsAny	16	False	2.033 ns
IndexOfAny	16	True	3.256 ns
ContainsAny	16	True	2.465 ns
IndexOfAny	32	False	2.908 ns
ContainsAny	32	False	2.613 ns
IndexOfAny	32	True	3.909 ns
ContainsAny	32	True	2.283 ns
IndexOfAny	64	False	3.445 ns
ContainsAny	64	False	3.155 ns
IndexOfAny	64	True	3.261 ns
ContainsAny	64	True	1.942 ns

API Proposal

namespace System;

public static class MemoryExtensions
{
    // Existing
    public static int IndexOf<T>(this ReadOnlySpan<T> span, T value) where T : IEquatable<T>?;
    public static bool Contains<T>(this ReadOnlySpan<T> span, T value) where T : IEquatable<T>?;
    public static int IndexOfAny<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static int IndexOfAnyExcept<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static int IndexOfAny(this ReadOnlySpan<char> span, SearchValues<string> values);

    // Proposed
    public static bool ContainsAny<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAny(this ReadOnlySpan<char> span, SearchValues<string> values);
}

API Usage

-if (text.IndexOfAny(s_invalidChars) >= 0)
+if (text.ContainsAny(s_invalidChars))

-if (text.IndexOfAnyExcept(s_allowedChars) >= 0)
+if (text.ContainsAnyExcept(s_allowedChars))

Alternative Designs

IndexOfAny has many overloads and variants. Should any of them also have ContainsAny variants?

static bool ContainsAny{Except}(this ReadOnlySpan<T> span, T value0, T value1);
static bool ContainsAny{Except}(this ReadOnlySpan<T> span, T value0, T value1, value2);
static bool ContainsAny{Except}(this ReadOnlySpan<T> span, ReadOnlySpan<T> values);
static bool ContainsAny{Except}InRange(this ReadOnlySpan<T> span, T min, T max);

static bool ContainsNewLine(ReadOnlySpan<char> value) =>
-   value.IndexOfAny('\r', '\n') >= 0;
+   value.ContainsAny('\r', '\n');

Risks

More methods to think about - ContainsAny/ContainsAnyExcept would be a new set of overloads on a common type.

Author:	MihaZupan
Assignees:	-
Labels:	api-suggestion, area-System.Memory
Milestone:	-

[stephentoub]

Which of the places we're currently using SearchValues with IndexOfAny{Except} would be replaceable/improved with the proposed methods?

[MihaZupan]

In System.Net:

• runtime/src/libraries/System.Net.Http/src/System/Net/Http/HttpRuleParser.cs

  Lines 43 to 47 in 37c2751

  	internal static bool IsToken(ReadOnlySpan<char> input) =>
  	input.IndexOfAnyExcept(s_tokenChars) < 0;
  	internal static bool IsToken(ReadOnlySpan<byte> input) =>
  	input.IndexOfAnyExcept(s_tokenBytes) < 0;

• runtime/src/libraries/System.Net.Primitives/src/System/Net/Cookie.cs

  Line 353 in 37c2751

  m_name.AsSpan().IndexOfAny(s_reservedToNameChars) >= 0)

• runtime/src/libraries/System.Net.Primitives/src/System/Net/Cookie.cs

  Line 568 in 37c2751

  name.AsSpan().IndexOfAnyExcept(s_domainChars) < 0;

• runtime/src/libraries/System.Private.Uri/src/System/DomainNameHelper.cs

  Line 209 in 37c2751

  if (asciiForm.AsSpan().IndexOfAny(s_unsafeForNormalizedHostChars) >= 0)

• runtime/src/libraries/System.Private.Uri/src/System/UriBuilder.cs

  Line 265 in 37c2751

  if (input.AsSpan().IndexOfAny(s_userInfoReservedChars) < 0)

• runtime/src/libraries/Common/src/System/Net/Security/TargetHostNameHelper.cs

  Line 17 in 37c2751

  name.IndexOfAnyExcept(s_safeDnsChars) < 0;

• runtime/src/libraries/System.Net.Http/src/System/Net/Http/MultipartContent.cs

  Line 85 in 37c2751

  if (boundary.AsSpan().IndexOfAnyExcept(s_allowedBoundaryChars) >= 0)

• runtime/src/libraries/System.Net.Mail/src/System/Net/Mail/MailBnfHelper.cs

  Line 149 in 37c2751

  if (data.Length == 0 || data.AsSpan().IndexOfAnyExcept(s_charactersAllowedInHeaderNames) >= 0)

• runtime/src/libraries/System.Net.HttpListener/src/System/Net/Managed/HttpListenerRequest.Managed.cs

  Line 87 in 37c2751

  if (_method.AsSpan().IndexOfAnyExcept(s_validMethodChars) >= 0)

Other:

• 4x in this file:

  runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/CompareInfo.Icu.cs

  Line 107 in 37c2751

  if (target.IndexOfAnyExcept(s_nonSpecialAsciiChars) >= 0)

• runtime/src/libraries/System.Private.DataContractSerialization/src/System/Xml/XmlConverter.cs

  Lines 1087 to 1091 in 37c2751

  	public static bool IsWhitespace(ReadOnlySpan<char> chars) =>
  	chars.IndexOfAnyExcept(s_whitespaceChars) < 0;
  	public static bool IsWhitespace(ReadOnlySpan<byte> bytes) =>
  	bytes.IndexOfAnyExcept(s_whitespaceBytes) < 0;

• runtime/src/libraries/Microsoft.Extensions.FileProviders.Physical/src/Internal/PathUtils.cs

  Lines 31 to 39 in 37c2751

  	internal static bool HasInvalidPathChars(string path)
  	{
  	return path.AsSpan().IndexOfAny(_invalidFileNameChars) >= 0;
  	}
  	internal static bool HasInvalidFilterChars(string path)
  	{
  	return path.AsSpan().IndexOfAny(_invalidFilterChars) >= 0;
  	}

• runtime/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/Base64Transforms.cs

  Line 243 in 37c2751

  if (inputBuffer.IndexOfAny(s_whiteSpace) >= 0)

• runtime/src/libraries/System.Text.Json/src/System/Text/Json/Reader/JsonReaderHelper.cs

  Line 21 in 37c2751

  => text.IndexOfAny(s_specialCharacters) >= 0;

• runtime/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/X500NameEncoder.cs

  Line 128 in 37c2751

  rdnValue.IndexOfAny(s_needsQuotingChars) >= 0;

AspNetCore (some places would likely switch from IndexOfAny to ContainsAny on the fast + IndexOfAny on the slow path):

• https://github.com/dotnet/aspnetcore/blob/c760054e1fbc8749bc0c6f2116b22532424ff60f/src/Middleware/HttpOverrides/src/ForwardedHeadersMiddleware.cs#L248
• https://github.com/dotnet/aspnetcore/blob/c760054e1fbc8749bc0c6f2116b22532424ff60f/src/Servers/Kestrel/Core/src/Internal/Http/HttpHeaders.cs#L293C32-L294
• https://github.com/dotnet/aspnetcore/blob/c760054e1fbc8749bc0c6f2116b22532424ff60f/src/Servers/Kestrel/Core/src/Internal/Http/HttpHeaders.cs#L282-L284
• 5x in this file: https://github.com/dotnet/aspnetcore/blob/c760054e1fbc8749bc0c6f2116b22532424ff60f/src/Http/Routing/src/Patterns/RoutePatternFactory.cs#L763
• https://github.com/dotnet/aspnetcore/blob/c760054e1fbc8749bc0c6f2116b22532424ff60f/src/Http/Routing/src/Patterns/RoutePatternParser.cs#L424
• Probably in some other place not yet using SearchValues - Use SearchValues throughout dotnet/aspnetcore aspnetcore#46484

[stephentoub]

Thanks. So a reasonable percentage of our current IndexOfAny use with SearchValues could now use ContainsAny. Would we be able to share most of the code in the SearchValues implementations?

[MihaZupan]

By abusing generics some more, yes: MihaZupan@2dec12f#diff-1616491faf2fad45e66a3301f2ed62e37e1c6454605ee94e2210913d58b32988
(that is if we wanted a specialized implementation at all vs. just the convenience of the API)

[bartonjs]

Video

• We need to overload across ReadOnlySpan and Span, since extension methods don't apply to implicit conversions.
• We decided to go ahead and square the circle and approve a Contains version of everything that has an IndexOfAny

namespace System;

public static partial class MemoryExtensions
{
    public static bool ContainsAny<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAny(this ReadOnlySpan<char> span, SearchValues<string> values);

    public static bool ContainsAny<T>(this Span<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAnyExcept<T>(this Span<T> span, SearchValues<T> values) where T : IEquatable<T>?;
    public static bool ContainsAny(this Span<char> span, SearchValues<string> values);

    // If any overloads of IndexOfAny{Except} are missing, consider them approved, too.
    // NOTE: Generic constraints are missing in these methods, because they were not reflected in the proposal.
    // The constraints should match the IndexOfAny versions
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, T value);
    public static bool ContainsAny<T>(this ReadOnlySpan<T> span, T value0, T value1);
    public static bool ContainsAny<T>(this ReadOnlySpan<T> span, T value0, T value1, T value2);
    public static bool ContainsAny<T>(this ReadOnlySpan<T> span, ReadOnlySpan<T> values);
    public static bool ContainsAnyInRange<T>(this ReadOnlySpan<T> span, T lowInclusive, T highInclusive);
    public static bool ContainsAnyWhiteSpace(this ReadOnlySpan<char> span);
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, T value0, T value1);
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, T value0, T value1, T value2);
    public static bool ContainsAnyExcept<T>(this ReadOnlySpan<T> span, ReadOnlySpan<T> values);
    public static bool ContainsAnyExceptInRange<T>(this ReadOnlySpan<T> span, T lowInclusive, T highInclusive);
    public static bool ContainsAnyExceptWhiteSpace(this ReadOnlySpan<char> span);

    public static bool ContainsAnyExcept<T>(this Span<T> span, T value);
    public static bool ContainsAny<T>(this Span<T> span, T value0, T value1);
    public static bool ContainsAny<T>(this Span<T> span, T value0, T value1, T value2);
    public static bool ContainsAny<T>(this Span<T> span, ReadOnlySpan<T> values);
    public static bool ContainsAnyInRange<T>(this Span<T> span, T lowInclusive, T highInclusive);
    public static bool ContainsAnyExcept<T>(this Span<T> span, T value0, T value1);
    public static bool ContainsAnyExcept<T>(this Span<T> span, T value0, T value1, T value2);
    public static bool ContainsAnyExcept<T>(this Span<T> span, ReadOnlySpan<T> values);
    public static bool ContainsAnyExceptInRange<T>(this Span<T> span, T lowInclusive, T highInclusive);

}