Perf optimization of module scanning for generic cycle detection #87532
Conversation
As part of investigation of the bug dotnet#66079 and implementation of the fix dotnet#71426 I noticed that one aggravating factor is that Crossgen2 starts analyzing the same assembly multiple times on its various parallel threads; this busywork additionally makes the app compete for access to the same metadata, making the initial analysis even slower. In accordance with Michal's suggestion from the PR thread dotnet#71426 I propose to modify the generic cycle detector to run the initial analysis single-threaded if the module in question is expected to be "generics-heavy" using the number of TypeSpec rows in its ECMA metadata as an indicator of module generic complexity. I have written a simple managed app scanning all runtime framework assemblies, ASP.NET assemblies and assemblies used in internal CoreCLR testing. I found out that the largest number of TypeSpec rows is in FSharp.Core (3855), followed by Microsoft.CodeAnalysis (3148). Based on these findings I have set the initial value of the cutoff for single-threaded analysis to 5000. Thanks Tomas
|
With this change, compilation of the repro case from #66079 goes down from about 8 to 2 minutes (with |
| } | ||
| else | ||
| { | ||
| lock (ownerModule) |
There was a problem hiding this comment.
We'd need to remember that nobody else should lock on EcmaModules so that we don't get an accidental deadlock somewhere.
I don't have strong opinions about this - the compilation will finish even without doing this and whether it take 2 minutes or 8 minutes doesn't make a huge difference - it's already orders of magnitude longer than usual - the assembly is simply AOT hostile and it will cause trouble for any AOT we have in .NET (NativeAOT, Mono AOT, crossgen2). I would be fine won't fixing that part.
|
Closing as there is no clear benefit in taking this change - it only covers a niche case that is unfriendly to AOT compilation one way or the other. |
ghost
locked as resolved and limited conversation to collaborators
2 participants
As part of investigation of the bug #66079 and implementation of the fix #71426 I noticed that one aggravating factor is that Crossgen2 starts analyzing the same assembly multiple times on its various parallel threads; this busywork additionally makes the app compete for access to the same metadata, making the initial analysis even slower.
In accordance with Michal's suggestion from the PR thread #71426 I propose to modify the generic cycle detector to run the initial analysis single-threaded if the module in question is expected to be "generics-heavy" using the number of TypeSpec rows in its ECMA metadata as an indicator of module generic complexity.
I have written a simple managed app scanning all runtime framework assemblies, ASP.NET assemblies and assemblies used in internal CoreCLR testing. I found out that the largest number of TypeSpec rows is in FSharp.Core (3855), followed by Microsoft.CodeAnalysis (3148). Based on these findings I have set the initial value of the cutoff for single-threaded analysis to 5000.
Thanks
Tomas
/cc @dotnet/crossgen-contrib