Add JMH Benchmarks

[tylerbenson]

Unfortunately there are some incompatibilities with how gradle is forking the jmh task. It includes multiple different versions of groovy, which causes an error. I worked around this by creating a separate task.

Sample run with and without the javaagent line commented out:

Benchmark                                            Mode  Cnt     Score   Units
Baseline:
ClassRetransformingBenchmark.testDeepRetransform     avgt    2   1502.486   us/op
ClassRetransformingBenchmark.testIgnoredRetransform  avgt    2   2077.988   us/op
ClassRetransformingBenchmark.testSimpleRetransform   avgt    2   1295.614   us/op

With Agent:
ClassRetransformingBenchmark.testDeepRetransform     avgt    2  16591.865   us/op
ClassRetransformingBenchmark.testIgnoredRetransform  avgt    2   4741.987   us/op
ClassRetransformingBenchmark.testSimpleRetransform   avgt    2  16515.228   us/op

[realark]

Is this related to #162?

[realark]

Looks good! Can you add instructions for running the benchmarks in benchmark/README.md? I assume it's something like this:

No Agent:

./gradlew benchmark:jmhAgent

With Agent:

// add -javaagent jvm arg by uncommenting benchmark.gradle:55
./gradlew benchmark:jmhAgent

[tylerbenson]

This PR is actually not dependent on the gradle upgrade PR. I initially did that as a troubleshooting step, but was able to work around it. I still think it's valuable to upgrade though.

I've modified the test to be able to test across many different agent versions. There is a significant regression in 0.2.7 which is probably caused by running two different byte transformers concurrently that I hope will be reduced when we remove byteman:

Benchmark                                                                Mode  Cnt      Score     Error  Units
ClassRetransformingBenchmark.WithAgent.testSimpleRetransform             avgt   20  20878.179 ± 677.773  us/op
ClassRetransformingBenchmark.WithAgent0210.testSimpleRetransform         avgt   20  20327.871 ± 263.895  us/op
ClassRetransformingBenchmark.WithAgent022.testSimpleRetransform          avgt   20   4437.608 ± 172.542  us/op
ClassRetransformingBenchmark.WithAgent024.testSimpleRetransform          avgt   20   5304.755 ± 219.633  us/op
ClassRetransformingBenchmark.WithAgent026.testSimpleRetransform          avgt   20   5343.443 ± 163.164  us/op
ClassRetransformingBenchmark.WithAgent027.testSimpleRetransform          avgt   20  23182.847 ± 646.143  us/op
ClassRetransformingBenchmark.WithAgent028.testSimpleRetransform          avgt   20  22964.934 ± 608.933  us/op
ClassRetransformingBenchmark.WithAgent029.testSimpleRetransform          avgt   20  20090.491 ± 413.056  us/op
ClassRetransformingBenchmark.testSimpleRetransform                       avgt   20   2062.694 ± 510.006  us/op

The stack profiling seems to indicate that byteman is still a significant point of overhead:

 10.9%  32.4% sun.instrument.InstrumentationImpl.retransformClasses0
  7.0%  20.8% sun.instrument.TransformerManager.transform
  2.9%   8.5% org.jboss.byteman.agent.Transformer.disableTriggers
  2.2%   6.6% sun.instrument.InstrumentationImpl.retransformClasses
  2.2%   6.6% org.jboss.byteman.rule.Rule.disableTriggersInternal
  1.8%   5.2% org.jboss.byteman.rule.type.TypeHelper.internalizeClass
  1.2%   3.5% java.lang.ThreadLocal$ThreadLocalMap.access$100
  1.0%   2.8% java.lang.ThreadLocal.set
  0.9%   2.6% org.jboss.byteman.agent.Transformer.transform
  0.3%   0.9% java.lang.String.startsWith
  3.4%  10.0% <other>

[realark]

Very interesting! We'll have to keep an eye on these as we do releases.