Description
I've just measured the overhead of the incremental compiler for the compiler subproject in scala/scala. The hot performance of the direct use of the compiler is 21033.036 ± 618.165 ms/op, whereas a loop of:
sbt> consoleProject
scala> for (i <- 1 to 100) { (clean in compiler).eval; update.eval; println(timeMillis((compile in compiler in Compile).eval)) }
Doesn't get faster than 36600ms (a 1.75x slowdown). I'm using a 2G heap
I'm using https://gist.github.com/retronym/9861cf798339f1fe6da77bbd0d742f75 to export the compiler options to directly use in scalac, and them retronym/scala-jmh-suite to measure the direct performance:
sbt> compilation/jmh:run HotScalacBenchmark -f3 -wi 2 -i 4 -jvmArgs -Xmx2G -p _scalaVersion=2.12.0-RC1-be43eb5 -p source=@/code/scala/sandbox/args-compile.txt
...
[info] Benchmark (_scalaVersion) (extraArgs) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile 2.12.0-RC1-be43eb5 @/code/scala/sandbox/args-compile.txt sample 12 21033.036 ± 618.165 ms/op
For 2.11.8, i see 23018ms direct, 31894 via SBT, a 1.38x slowdown.
Incidentally, according both 2.11.8 and 2.12 compile the compiler at 2800LOC/s (non comment/blank). The code base in smaller in 2.12 because we've removed the legacy backend, GenICode.
These slowdown mirror what I saw for a smaller project (the sources of scalap in retronym/scala-jmh-suite corpus/scalap), a 2x slowdown in hot performance.
https://gist.github.com/retronym/7d17209bf51d8a46d1a2592d659000fb
@milessabin also reported the same sort of slowdowns in shapeless:
Compiling with plain scalac, command line similar to yours above I see,
2.11.8 129s
2.12.x 155s
So 1.2x in this case ... does that line up with what you're seeing?
In SBT I'm seeing,
2.11.8 138s
2.12.x 237s
ie. 1.7x, which suggests that your hypothesis that there's a regression in the incremental compiler might be right.