Optimize ImmutableHashSet<T>.IsProperSubsetOf to avoid unnecessary allocations

[aw0lid]

Part of #127279

Summary

ImmutableHashSet<T>.IsProperSubsetOf always creates a new intermediate HashSet<T> for the other collection, leading to avoidable allocations and GC pressure, especially for large datasets

Optimization Logic

• O(1) Pre-Scan: Immediately returns false if other is an ICollection with a smaller or equal Count. By performing this validation upfront, the need for tracking variables like matches and extraFound is eliminated, as any complete match is now mathematically guaranteed to be a proper subset.

• Fast-Path Pattern Matching: Detects ImmutableHashSet<T> and HashSet<T> to bypass intermediate allocations.

• Comparer Guard: Validates EqualityComparer compatibility before triggering fast paths to ensure logical consistency.

• Short-Circuit Validation: Re-validates Count within specialized paths for an immediate exit before $O(n)$ enumeration.

• Don't repeat your self: reused SetEqualsWithHashset and SetEqualsWithImmutableHashset methods to avoid code duplication while ensuring we leverage the $O(1)$ lookup efficiency when other is a Hashset<T>.

• Zero-Allocation Execution: Direct iteration over compatible collections, eliminating the costly new HashSet<T>(other) fallback.

• Deferred fallback: Reserves the expensive allocation solely for general IEnumerable types.

Click to expand Benchmark Source Code

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Order;
using BenchmarkDotNet.Running;
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;

namespace ImmutableHashSetBenchmarks
{
    [MemoryDiagnoser]
    [Orderer(SummaryOrderPolicy.FastestToSlowest)]
    [RankColumn]
    public class ImmutableHashSetIsProperSubsetOfBenchmark
    {
        private ImmutableHashSet<int> _sourceSet = null!;
        private ImmutableHashSet<int> _immutableLarger = null!;
        private HashSet<int> _bclHashSetLarger = null!;
        private List<int> _listLarger = null!;
        private int[] _arrayLarger = null!;

        private ImmutableHashSet<int> _immutableSmaller = null!;
        private ImmutableHashSet<int> _immutableSameCount = null!;
        
        private HashSet<int> _bclHashSetLargerDiffComparer = null!;

        private List<int> _listWithDuplicatesButProper = null!;
        private ImmutableHashSet<int> _emptySource = null!;
        private List<int> _listSameElementsWithDuplicates = null!;

        [Params(100000)]
        public int Size { get; set; }

        [GlobalSetup]
        public void Setup()
        {
            var elements = Enumerable.Range(0, Size).ToList();
            var largerElements = Enumerable.Range(0, Size + 10).ToList();
            var smallerElements = Enumerable.Range(0, Size - 10).ToList();
            var reverseComparer = new ReverseComparer<int>();

            _sourceSet = ImmutableHashSet.CreateRange(elements);
            
            _immutableLarger = ImmutableHashSet.CreateRange(largerElements);
            _bclHashSetLarger = new HashSet<int>(largerElements);
            _listLarger = largerElements;
            _arrayLarger = largerElements.ToArray();

            _immutableSmaller = ImmutableHashSet.CreateRange(smallerElements);
            _immutableSameCount = ImmutableHashSet.CreateRange(elements);

            _bclHashSetLargerDiffComparer = new HashSet<int>(largerElements, reverseComparer);

            _listWithDuplicatesButProper = elements.Concat(new[] { Size + 1, Size + 1, Size + 1 }).ToList();
            _emptySource = ImmutableHashSet<int>.Empty;
            _listSameElementsWithDuplicates = elements.Concat(elements).ToList();
        }

        #region Fast Path: Same Type and Comparer (Optimized)

        [Benchmark(Description = "ImmutableHashSet (Proper Subset - O(N))")]
        public bool Case_ImmutableHashSet_Proper() => _sourceSet.IsProperSubsetOf(_immutableLarger);

        [Benchmark(Description = "BCL HashSet (Proper Subset - O(N))")]
        public bool Case_BclHashSet_Proper() => _sourceSet.IsProperSubsetOf(_bclHashSetLarger);

        #endregion

        #region Early Exit: Count Check (O(1))

        [Benchmark(Description = "Empty Source (O(1) Check)")]
        public bool Case_EmptySource_Proper() => _emptySource.IsProperSubsetOf(_bclHashSetLarger);

        [Benchmark(Description = "List (Same Elements with Duplicates - Not Proper)")]
        public bool Case_List_Duplicates_NotProper() => _sourceSet.IsProperSubsetOf(_listSameElementsWithDuplicates);

        [Benchmark(Description = "Early Exit (Other is Smaller)")]
        public bool Case_SmallerCount() => _sourceSet.IsProperSubsetOf(_immutableSmaller);

        [Benchmark(Description = "Early Exit (Same Count - Cannot be Proper)")]
        public bool Case_SameCount() => _sourceSet.IsProperSubsetOf(_immutableSameCount);

        #endregion

        #region Fallback Path: Non-Set or Different Comparer

        [Benchmark(Description = "List (Proper - Fallback to HashSet)")]
        public bool Case_List_Proper() => _sourceSet.IsProperSubsetOf(_listLarger);

        [Benchmark(Description = "Array (Proper - Fallback to HashSet)")]
        public bool Case_Array_Proper() => _sourceSet.IsProperSubsetOf(_arrayLarger);

        [Benchmark(Description = "HashSet (Different Comparer - Force Fallback)")]
        public bool Case_HashSet_DiffComparer() => _sourceSet.IsProperSubsetOf(_bclHashSetLargerDiffComparer);

        [Benchmark(Description = "List with Duplicates (Proper Subset)")]
        public bool Case_List_Duplicates_Proper() => _sourceSet.IsProperSubsetOf(_listWithDuplicatesButProper);

        #endregion
    }

    public class ReverseComparer<T> : IEqualityComparer<T> where T : IComparable<T>
    {
        public bool Equals(T? x, T? y) => x?.CompareTo(y) == 0;
        public int GetHashCode(T? obj) => obj?.GetHashCode() ?? 0;
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            BenchmarkRunner.Run<ImmutableHashSetIsProperSubsetOfBenchmark>();
        }
    }
}

Click to expand Benchmark Results

Benchmark Results (Before Optimization)

Method	Size	Mean	Error	StdDev	Rank	Gen0	Gen1	Gen2	Allocated
'Empty Source (O(1) Check)'	100000	3.020 ns	0.0522 ns	0.0463 ns	1	-	-	-	-
'List (Same Elements with Duplicates - Not Proper)'	100000	2,253,302.064 ns	43,996.1710 ns	83,707.2944 ns	2	85.9375	85.9375	85.9375	3605725 B
'BCL HashSet (Proper Subset - O(N))'	100000	7,143,590.897 ns	55,965.9984 ns	52,350.6297 ns	3	62.5000	62.5000	62.5000	1738869 B
'Array (Proper - Fallback to HashSet)'	100000	7,191,053.721 ns	60,956.8792 ns	54,036.6857 ns	3	70.3125	70.3125	70.3125	1738731 B
'Early Exit (Other is Smaller)'	100000	7,252,623.997 ns	127,108.7726 ns	112,678.6161 ns	3	70.3125	70.3125	70.3125	1738868 B
'Early Exit (Same Count - Cannot be Proper)'	100000	7,254,561.478 ns	73,899.0961 ns	57,695.5534 ns	3	78.1250	78.1250	78.1250	1738874 B
'List (Proper - Fallback to HashSet)'	100000	8,628,967.714 ns	103,114.2276 ns	96,453.1125 ns	4	78.1250	78.1250	78.1250	1738861 B
'List with Duplicates (Proper Subset)'	100000	8,979,029.530 ns	124,859.7911 ns	104,263.5805 ns	4	78.1250	78.1250	78.1250	1738861 B
'HashSet (Different Comparer - Force Fallback)'	100000	9,236,192.453 ns	94,804.2142 ns	88,679.9208 ns	4	78.1250	78.1250	78.1250	1738861 B
'ImmutableHashSet (Proper Subset - O(N))'	100000	14,980,671.116 ns	155,763.4001 ns	145,701.1812 ns	5	78.1250	78.1250	78.1250	1738897 B

---

Benchmark Results (After Optimization)

Method	Size	Mean	Error	StdDev	Rank	Gen0	Gen1	Gen2	Allocated
'Empty Source (O(1) Check)'	100000	1.769 ns	0.0372 ns	0.0348 ns	1	-	-	-	-
'Early Exit (Same Count - Cannot be Proper)'	100000	2.261 ns	0.0409 ns	0.0363 ns	2	-	-	-	-
'Early Exit (Other is Smaller)'	100000	2.401 ns	0.0800 ns	0.0748 ns	2	-	-	-	-
'List (Same Elements with Duplicates - Not Proper)'	100000	2,226,632.494 ns	44,488.2292 ns	94,808.0507 ns	3	82.0313	82.0313	82.0313	3605636 B
'Array (Proper - Fallback to HashSet)'	100000	4,128,214.156 ns	40,633.6128 ns	36,020.6393 ns	4	62.5000	62.5000	62.5000	1738710 B
'HashSet (Different Comparer - Force Fallback)'	100000	4,310,472.716 ns	39,782.7924 ns	33,220.4335 ns	5	70.3125	70.3125	70.3125	1738810 B
'BCL HashSet (Proper Subset - O(N))'	100000	5,622,467.743 ns	43,428.7884 ns	36,265.0052 ns	6	-	-	-	-
'List (Proper - Fallback to HashSet)'	100000	6,933,851.438 ns	51,688.7203 ns	43,162.4224 ns	7	62.5000	62.5000	62.5000	1738734 B
'List with Duplicates (Proper Subset)'	100000	7,455,423.480 ns	92,394.8283 ns	81,905.6087 ns	8	70.3125	70.3125	70.3125	1738817 B
'ImmutableHashSet (Proper Subset - O(N))'	100000	13,207,980.537 ns	130,270.0225 ns	115,480.9819 ns	9	-	-	-	-

Performance Analysis Summary (100,000 Elements)

Case / Method	Before (ns)	After (ns)	Speedup Ratio	Memory Improvement
Early Exit (Other is Smaller)	7,252,623	2.401	~3,020,667x	-100% (Zero Alloc)
Early Exit (Same Count)	7,254,561	2.261	~3,208,563x	-100% (Zero Alloc)
Empty Source	3.020	1.769	1.71x	Zero Alloc
BCL HashSet (Proper Subset)	7,143,590	5,622,467	1.27x	-100% (Zero Alloc)
List (Duplicates - Not Proper)	2,253,302	2,226,632	1.01x	Stable (3.6 MB)
Array (Fallback to HashSet)	7,191,053	4,128,214	1.74x	Stable (1.7 MB)
List (Proper - Fallback)	8,628,967	6,933,851	1.24x	Stable (1.7 MB)
List with Duplicates (Proper)	8,979,029	7,455,423	1.20x	Stable (1.7 MB)
HashSet (Diff Comparer)	9,236,192	4,310,472	2.14x	Stable (1.7 MB)
ImmutableHashSet (Proper)	14,980,671	13,207,980	1.13x	-100% (Zero Alloc)

---

✅ Unit Tests Added

Added unit tests for IsProperSubsetOf to cover various edge cases and ensure the correctness of the new logic:

• Mismatched Comparers: Validated behavior when comparing sets with different comparers (e.g., Ordinal vs. OrdinalIgnoreCase).
• Duplicate Elements: Verified ICollection<T> logic to ensure that collections with duplicates are handled correctly by the early exit (Count <= origin.Count).
• Empty Set Scenarios: Confirmed expected behavior when either the origin or the target collection is empty.
• Equality Logic: Covered cases where logical equality might differ from reference equality in comparers.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[aw0lid]

Hi everyone, just a gentle follow-up on this PR
CC/ @eiriktsarpalis, @MihaZupan