pin over SPMI has unexpectedly high variance

[jakobbotsch]

Of 10 PIN runs of sequential SPMI over librares.pmi I got the following sorted instruction counts:

315928112108
315937618484
316056486869
316226394292
316333895754
316557609181
316704720271
316791621329
317697057755
317735093948

The largest number here is 0.6% higher than the lowest number.

As Andy pointed out there will be some variance simply due to various factors of randomization (e.g. ASLR), but I still think this is an unexpectedly high variance given that PIN uses instrumentation.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Of 10 PIN runs of SPMI over librares.pmi I got the following sorted instruction counts:

315928112108
315937618484
316056486869
316226394292
316333895754
316557609181
316704720271
316791621329
317697057755
317735093948

The largest number here is 0.6% higher than the lowest number.

As Andy pointed out there will be some variance simply due to various factors of randomization (e.g. ASLR), but I still think this is an unexpectedly high variance given that PIN uses instrumentation.

Author:	jakobbotsch
Assignees:	-
Labels:	area-CodeGen-coreclr, untriaged
Milestone:	-

[jakobbotsch]

I've created a new pintool (well, modified the inscount_vregs example part of PIN) that only instruments clrjit. By default the example instruments all code, which includes the SPMI replay layer. The new counts are:

231591325570
231591520643
231591701976
231591746231
231592022315
231603667317
231603669026
231603675339
231603849596
231604019663

This is a difference of 0.005% between the largest and smallest value, which seems much more reasonable.
I'll talk to some people internally to figure out if/how we can distribute this for jit devs.

cc @dotnet/jit-contrib