NPE during compilation

[savulchik]

During compilation of production code using scala-async 0.9.3 and scala 2.11.7 I've got an exception:

java.lang.NullPointerException
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1695)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1386)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1366)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1742)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.api.Trees$Transformer$$anonfun$transformCaseDefs$1.apply(Trees.scala:2581)
        at scala.reflect.api.Trees$Transformer$$anonfun$transformCaseDefs$1.apply(Trees.scala:2581)
        at scala.collection.immutable.List.loop$1(List.scala:173)
        at scala.collection.immutable.List.mapConserve(List.scala:189)
        at scala.reflect.api.Trees$Transformer.transformCaseDefs(Trees.scala:2581)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1382)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1366)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1390)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1366)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1366)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1421)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.internal.Trees$$anonfun$itransform$2.apply(Trees.scala:1363)
        at scala.reflect.internal.Trees$$anonfun$itransform$2.apply(Trees.scala:1361)
        at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2600)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1360)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1404)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:2563)
        at scala.reflect.internal.Trees$$anonfun$itransform$4.apply(Trees.scala:1408)
        at scala.reflect.internal.Trees$$anonfun$itransform$4.apply(Trees.scala:1407)
        at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2600)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1406)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1366)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.internal.Trees$$anonfun$itransform$2.apply(Trees.scala:1363)
        at scala.reflect.internal.Trees$$anonfun$itransform$2.apply(Trees.scala:1361)
        at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2600)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1360)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1404)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:2563)
        at scala.reflect.internal.Trees$$anonfun$itransform$4.apply(Trees.scala:1408)
        at scala.reflect.internal.Trees$$anonfun$itransform$4.apply(Trees.scala:1407)
        at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2600)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1406)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStat(RefChecks.scala:1250)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer$$anonfun$transformStats$1.apply(RefChecks.scala:1165)
        at scala.collection.immutable.List.flatMap(List.scala:327)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:1165)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transformStats(RefChecks.scala:111)
        at scala.reflect.internal.Trees$$anonfun$itransform$7.apply(Trees.scala:1426)
        at scala.reflect.internal.Trees$$anonfun$itransform$7.apply(Trees.scala:1426)
        at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2600)
        at scala.reflect.internal.Trees$class.itransform(Trees.scala:1425)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:16)
        at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2555)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:1764)
        at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:111)
        at scala.tools.nsc.ast.Trees$Transformer.transformUnit(Trees.scala:147)
        at scala.tools.nsc.transform.Transform$Phase.apply(Transform.scala:30)
        at scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:440)
        at scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:431)
        at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:440)
        at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:398)
        at scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:398)
        at scala.collection.Iterator$class.foreach(Iterator.scala:742)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
        at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:398)
        at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
        at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1486)
        at scala.tools.nsc.Global$Run.compileSources(Global.scala:1481)
        at scala.tools.nsc.Global$Run.compile(Global.scala:1582)
        at xsbt.CachedCompiler0.run(CompilerInterface.scala:116)
        at xsbt.CachedCompiler0.run(CompilerInterface.scala:95)
        at xsbt.CompilerInterface.run(CompilerInterface.scala:26)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:101)
        at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:47)
        at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
        at sbt.compiler.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply$mcV$sp(MixedAnalyzingCompiler.scala:51)
        at sbt.compiler.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply(MixedAnalyzingCompiler.scala:51)
        at sbt.compiler.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply(MixedAnalyzingCompiler.scala:51)
        at sbt.compiler.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:75)
        at sbt.compiler.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:50)
        at sbt.compiler.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:65)
        at sbt.compiler.IC$$anonfun$compileInternal$1.apply(IncrementalCompiler.scala:160)
        at sbt.compiler.IC$$anonfun$compileInternal$1.apply(IncrementalCompiler.scala:160)
        at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:66)
        at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:64)
        at sbt.inc.IncrementalCommon.cycle(IncrementalCommon.scala:31)
        at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:62)
        at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:61)
        at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:89)
        at sbt.inc.Incremental$.compile(Incremental.scala:61)
        at sbt.inc.IncrementalCompile$.apply(Compile.scala:54)
        at sbt.compiler.IC$.compileInternal(IncrementalCompiler.scala:160)
        at sbt.compiler.IC$.incrementalCompile(IncrementalCompiler.scala:138)
        at sbt.Compiler$.compile(Compiler.scala:128)
        at sbt.Compiler$.compile(Compiler.scala:114)
        at sbt.Defaults$.sbt$Defaults$$compileIncrementalTaskImpl(Defaults.scala:814)
        at sbt.Defaults$$anonfun$compileIncrementalTask$1.apply(Defaults.scala:805)
        at sbt.Defaults$$anonfun$compileIncrementalTask$1.apply(Defaults.scala:803)
        at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
        at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
        at sbt.std.Transform$$anon$4.work(System.scala:63)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
        at sbt.Execute.work(Execute.scala:235)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
        at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

cc @retronym

[savulchik]

To workaround the issue I had to restructure production code as mentioned in docs.

Before this analysis and transformation, the program is normalized into a form amenable to this manipulation. This is called the "A Normal Form" (ANF), and roughly means that:
if and match constructs are only used as statements; they cannot be used as an expression.

I had match expression with assignment in async block

async {
...
val bar = foo match { case ... => await(...) }
...
}

So I removed the assignment and left match expression as the last expression of async block.

[retronym]

@savulchik Did you encounter this in newly written code, or was the bug uncovered in previously working code by upgrading to Scala 2.11.7?

Could you share the before/after code in a bit more detail?

[savulchik]

@retronym in newly written code. I'm able to reproduce the problem using scala 2.11.6 too.

Unfortunately I can't share production code. Later I'll do another attempt to arrange sample project.

[savulchik]

@retronym here is a sample project to reproduce the problem https://github.com/savulchik/scala-async-bug

[retronym]

Thanks for the test project. I'll release 0.9.4 this week with a fix.