Avoid generic virtual dispatch for frozen collections alternate lookup #108732
+411
−104
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
In .NET 9 the Alternate Lookup concept was added to dictionaries where you could look up in a dictionary by a key type different than the original one. The main motivator was reading data off the wire e.g. keying a
Dictionary<string, T>
using aReadOnlySpan<char>
(orbyte
). Thus I created a PR in ASP.NET Core to use this new method onFrozenDictionary<string, T>
when matching routing but was surprised to find that there's quite a hefty penalty when using Alternate Lookups due to the generic virtual dispatch mechanism itself being a dictionary. Thus while the memory allocations disappeared the performance in terms of speed improved only slightly in the microbenchmarks.See dotnet/aspnetcore#58305
Solution
FrozenDictionary<string, T>
currently exposes a method with the following signature where the actual work happens.To replace it, a new mechanism was added:
The idea is that when the
AlternateLookup
struct is created it pays the price of the generic virtual dispatch once. It can then use the delegate as follows to avoid generic virtual dispatch:In concrete types of
FrozenDictionary
we can then use the following pattern to cache a static instance of the delegate which calls the actual worker method.In the above example
GetAlternateLookupDelegate<TAlternateKey>
simply loads a singleton delegate from a nested generic type. The delegate is implemented by callingGetValueRefOrNullRefCoreAlternate<TAlternateKey
which although generic importantly is not a virtual method.In order to avoid repeating this pattern for all the classes inheriting from
OrdinalStringFrozenDictionary
, a different pattern is used there. Note that in this case we do not need generics as we know that the concrete type ofTAlternateKey
will beReadOnlySpan<char>
, a fact which is asserted when creating the lookup.In the above example we are able to remove the generics from the worker method, and thus we can make it virtual - still avoiding the combination of generics and virtual dispatch. Thus the derived types can use the existing approach as follows.
Note that further investigation is necessary to measure whether it is a good idea to remove the virtual dispatch entirely even for classes inheriting
OrdinalStringFrozenDictionary
. It would result in more complex code but could be faster depending on how expensive the (non-generic) virtual dispatch is.Conclusion
As we can see from the benchmarks below, the PR does not change the "normal"
TryGetValue
scenario and improves the "Alternate Lookup"TryGetValue
scenario significantly.This way the alternate lookup still suffers a performance penalty but it's not quite as significant and is more likely to justify it's use if it results in fewer allocations.
Benchmark Results
dotnet/performance Frozen Dictionary benchmarks (https://github.com/dotnet/performance/blob/main/src/benchmarks/micro/libraries/System.Collections/Frozen/Perf_FrozenDictionary_String.cs) were modified locally to add a case to use the AlternateLookup feature. I didn't create a PR as I couldn't find a clean way to add it with conditional compilation for .NET 9+
DefaultFrozenDictionary
LengthBucketsFrozenDictionary
SingleCharFrozenDictionary
SubstringFrozenDictionary