Potential performance regression when deriving large case classess

[WojciechMazur]

Based on failure in Open Community Build #2826 in library https://github.com/sirthias/borer

Scala 3.1.3 builds were stable when compiling derivation tests using -Xss=2m, however, in Scala 3.2.0-RCx they end up with StackOverflowError in the staging phase, even with increased -Xss=4m

Compiler version

3.2.0-RC3

Minimized code

No code minimization yet, based on my observations project might fail with StackOverflowError when deriving encoder here for case class using 100 fields.

Output

Stacktrace (click arrow to expand)

[error] ## Exception when compiling 1 sources to /home/wmazur/projects/virtuslab/community-build3/repo/derivation/.jvm/target/scala-3.2.0-RC3/test-classes
[error] java.lang.StackOverflowError
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1404)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1412)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1418)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1430)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1432)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:49)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:132)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform$$anonfun$1(Trees.scala:1513)
[error]         at scala.collection.immutable.List.mapConserve(List.scala:472)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1513)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1418)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:58)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1481)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1432)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:49)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:132)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock$$anonfun$1$$anonfun$1(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1199)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1432)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:49)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:132)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
.... // Last 20 lines repeat ~50 times in the stacktrace
[error] (derivation / Test / compileIncremental) java.lang.StackOverflowError

Expectation

It should be checked whether new versions of compiler does not imply significantly larger memory usage

[prolativ]

@WojciechMazur The Community Build link seems wrong to me - that is a build of https://github.com/valskalla/odin and it was successful

[WojciechMazur]

@prolativ Maybe I've copied the wrong link. I've updated the link above. It now points to the latest, still failing, build of boerer.

[WojciechMazur]

Project is now passing the OpenCB, the issue might have been present due to testing infrastructure and default -Xss sizes