Guarded devirtualization based on whole program view

[MichalStrehovsky]

In Native AOT we have a whole program view that often knows the full set of types that could be implementing an interface. RyuJIT already takes advantage of it for the case of interfaces implemented by exactly one class.

It would be nice if RyuJIT could also do guarded devirtualization for the case of multiple implementing classes. I'm looking at the ASP.NET Stage1 Goldilocks app profiles where I see we're spending about 3% of samples in RhpInterfaceDispatch1. This helper performs cached interface dispatch on monomorphic callsites and looks like this:

00007FF787DFFC80 4D 8B 5A 08          mov         r11,qword ptr [r10+8]  
00007FF787DFFC84 48 8B 01             mov         rax,qword ptr [rcx]  
00007FF787DFFC87 49 3B 43 20          cmp         rax,qword ptr [r11+20h]  
00007FF787DFFC8B 75 04                jne         Attempt1 (07FF787DFFC91h)  
00007FF787DFFC8D 41 FF 63 28          jmp         qword ptr [r11+28h]  
00007FF787DFFC91 E9 CA 07 00 00       jmp         RhpInterfaceDispatchSlow (07FF787E00460h)  

The fact that we see 3% of samples in these 6 assembly instructions means that:

1. Monomorphic interface dispatch is very common in ASP.NET
2. We're getting guaranteed CPU pipeline stalls very often: there's only one such helper shared for all interface calls and it's getting guaranteed mispredicted every time more than one interface call is in the hot path.

I had a look at the places where this happens and there's often only 2-3 classes that implement the interface in the whole program. It would likely help a ton if RyuJIT could generate inline if/else if/else if/else checks for these. It would be nice if the number of supported targets would be configurable through --codegenopt so that we can tune the right value of it for BLENDED/SPEED_OPT settings.

Cc @EgorBo

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In Native AOT we have a whole program view that often knows the full set of types that could be implementing an interface. RyuJIT already takes advantage of it for the case of interfaces implemented by exactly one class.

It would be nice if RyuJIT could also do guarded devirtualization for the case of multiple implementing classes. I'm looking at the ASP.NET Stage1 Goldilocks app profiles where I see we're spending about 3% of samples in RhpInterfaceDispatch1. This helper performs cached interface dispatch on monomorphic callsites and looks like this:

00007FF787DFFC80 4D 8B 5A 08          mov         r11,qword ptr [r10+8]  
00007FF787DFFC84 48 8B 01             mov         rax,qword ptr [rcx]  
00007FF787DFFC87 49 3B 43 20          cmp         rax,qword ptr [r11+20h]  
00007FF787DFFC8B 75 04                jne         Attempt1 (07FF787DFFC91h)  
00007FF787DFFC8D 41 FF 63 28          jmp         qword ptr [r11+28h]  
00007FF787DFFC91 E9 CA 07 00 00       jmp         RhpInterfaceDispatchSlow (07FF787E00460h)  

The fact that we see 3% of samples in these 6 assembly instructions means that:

1. Monomorphic interface dispatch is very common in ASP.NET
2. We're getting guaranteed CPU pipeline stalls very often: there's only one such helper shared for all interface calls and it's getting guaranteed mispredicted every time more than one interface call is in the hot path.

I had a look at the places where this happens and there's often only 2-3 classes that implement the interface in the whole program. It would likely help a ton if RyuJIT could generate inline if/else if/else if/else checks for these. It would be nice if the number of supported targets would be configurable through --codegenopt so that we can tune the right value of it for BLENDED/SPEED_OPT settings.

Cc @EgorBo

Author:	MichalStrehovsky
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[jkotas]

We're getting guaranteed CPU pipeline stalls very often: there's only one such helper shared for all interface calls and it's getting guaranteed mispredicted every time more than one interface call is in the hot path.

Should we also do something about this for the general case while we are on it? We can change the sequence for interface calls to be a call to a helper that returns the target address (e.g. using a hash table), followed by indirect call. This would affect tunning of the guarded devirtualization.

(The current implementation of interface calls in native AOT does not scale well with the degree of polymorphism. It is only matter of time until somebody hits a super polymorphic call on a hot path and observes major perf gap.)

[MichalStrehovsky]

We're getting guaranteed CPU pipeline stalls very often: there's only one such helper shared for all interface calls and it's getting guaranteed mispredicted every time more than one interface call is in the hot path.

Should we also do something about this for the general case while we are on it? We can change the sequence for interface calls to be a call to a helper that returns the target address (e.g. using a hash table), followed by indirect call. This would affect tunning of the guarded devirtualization.

(The current implementation of interface calls in native AOT does not scale well with the degree of polymorphism. It is only matter of time until somebody hits a super polymorphic call on a hot path and observes major perf gap.)

Opened #86238 with something close to that (I didn't redo it to use a hash table but that one would likely be even worse for monomorphic callsites and start benefitting only heavily megamorphic callsites). It is a lot worse than the dispatch cell based approach.

[EgorBo]

For the multiple guarded devirtualization case I have a branch somewhere, just need to clean it up