Profile Synthesis Work Items

[AndyAyersMS]

This issue captures work items and other notes related to profile synthesis in RyuJit.

Overview

Profile synthesis is an algorithm to estimate plausible block and edge profile data. It can be used to guide optimizations in the absence of true profile data, fill in gaps in profile data, or repair damaged (inconsistent data).

See the section on synthesis in the main Dynamic PGO for more context.

Completed for .NET 8

• implement core reconstruction algorithm (JIT: initial implementation of profile synthesis #82926). This turns local likelihoods into global counts.
• extend to handle inlinee flow graphs (see JIT: it is difficult to do flow analysis on inlinees #82755)
• consistency check and fix issues (JIT: profile synthesis consistency checking and more #83068)
• PGO blend modes, random, etc JIT: profile synthesis blend and repair modes #83567
• Repair mode JIT: profile synthesis blend and repair modes #83567, JIT: extend initial profile repair to more cases #84741
• Proper weight comp for finallies (JIT: treat synthesized profile data as pgo data, fix finally weights #83185)
• Profile fill-in for OSR cases where Tier0 instrumented method never returns (JIT: extend initial profile repair to more cases #84741)

Opportunistic work for .NET 8

• come up with some strategy for irreducible loops (see notes below)
• assess how often $C_p$ gets capped in real programs (not many; see notes below)
• if we end up capping, can we go back and revise likelihoods so profile data is consistent (JIT: extend initial profile repair to more cases #84741, ...)
• faster way of doing fgGetPredForBlock -- we often want to go from a block to the successor edges
• vet heuristics against some real data -- maybe leverage MIBC compare capabilities.
  Some initial data in JIT: profile synthesis consistency checking and more #83068.
• IR based heuristics (perhaps)
• When computing $C_p$ avoid repeatedly propagating through nested loops
• Fake BB0 or always force scratch BB or just tolerate fgFirstBB being special
• Reconcile with our other loop finding stuff (Improve JIT loop optimizations #65342)
• Stop the upweight/downweight of loop blocks in rest of jit
• Durable edge properties (entry, exit, back)
• Tweak RunRarely to mean at or near zero (Consider fully replacing BBF_RUN_RARELY with the weight of 0 #48778)
• OSR entry weight still seems off
• Special handling for deep nests?

Note a number of the above were handled in .NET 9, see #93020

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue captures work items and other notes related to profile synthesis in RyuJit.

Overview

Profile synthesis is an algorithm to estimate plausible block and edge profile data. It can be used to guide optimizations in the absence of true profile data, fill in gaps in profile data, or repair damaged (inconsistent data).

See the section on synthesis in the main Dynamic PGO for more context.

Proposed work

• implement core reconstruction algorithm (JIT: initial implementation of profile synthesis #82926). This turns local likelihoods into global counts.
• extend to handle inlinee flow graphs (see Fatal error. Internal CLR error. (0x80131506) in .NET 7 during GC #82753)
• consistency check and fix issues
• come up with some strategy for irredicible loops (see notes below)
• assess how often $cp$ gets capped in real programs (see notes below)
• faster way of doing fgGetPredForBlock -- we often want to go from a block to the successor edges
• vet heuristics against some real data -- maybe leverage MIBC compare capabilities
• IR based heuristics (perhaps)
• PGO blend modes, random, etc
• Repair mode
• When computing $cp$ avoid repeatedly propagating through nested loops
• Fake BB0 or always force scratch BB or just tolerate fgFirstBB being special
• Reconcile with our other loop finding stuff
• Stop the upweight/downweight of loop blocks in rest of jit
• Durable edge properties (entry, exit, back)
• Proper weight comp for finallies
• Tweak RunRarely to mean at or near zero
• OSR entry weight still seems off
• Special handling for deep nests?

Author:	AndyAyersMS
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[AndyAyersMS]

Some initial data using aspnet.run collection:

• excluding methods with irreducible or capped profiles, synthesis results pass the profile checker, so are generally consistent
• 8 methods get capped. They all have loops where the loop exit is a throw. One such example:

https://github.com/aspnet/Security/blob/26d27d871b7992022c082dc207e3d126e1d9d278/shared/Microsoft.AspNetCore.ChunkingCookieManager.Sources/ChunkingCookieManager.cs#L111-L121

We can get rid of the capping by fixing the heuristic, but then we have a throw block with nonzero profile weight.

• Around 1000 methods have irreducible loops. Many of these are MoveNext async state machine methods. For example:
  

Note how there is a loop complex involving BB12, BB06, BB04, BB07, and BB11 but no single entry point.

We can try and handle these during synthesis, but the heuristics might not be great and it would require iteration. Or perhaps we can look into fixing the async codegen so the loops it produces are reducible.

[AndyAyersMS]

@VSadov who is the best person to ask these days about async method IL generation in Roslyn? I am wondering if the irreducible flow I am seeing (like in the example just above) is something that could easily be fixed with a bit of code duplication.

As is, the jit will not recognize the loops in these methods as loops and so a lot of the classic loop optimizations (say hoisting) won't fire. It may not matter in practice as these methods are tough to optimize anyways and maybe not that CPU intensive. But perhaps worth a closer look.

One crude way to view this is that if there were a source gen option for the async state machines, and that code spit had to resort to goto to express the desired control flow, it would quite possibly indicate irreducible flow.

[BruceForstall]

@AndyAyersMS Is the reference to #82753 incorrect?

[AndyAyersMS]

@AndyAyersMS Is the reference to #82753 incorrect?

Thanks -- fixed that above: the right issue is #82755.