NullPointerException in TASTy reader during compile when using Scala 3 modules from Scala 2.13.6 with scalacOptions += "-Ytasty-reader"

[wbillingsley]

reproduction steps

using Scala 3.0.0 and 2.13.6,

Pushed as a commit of a project, as it is hard for me to produce a more minimal example (I don't know which Scala 3 file the TASTy reader is NPE'ing on)
https://github.com/assessory/assessory/tree/581e12dead9e22d71240fd64ae531d10d8c37a65

sbt
project play
compile

problem

Produces this NPE from within the compiler:

[error] ## Exception when compiling 30 sources to C:\Users\wbilling\sourcecode\github\assessory\modules\play\target\scala-2.13\classes
[error] java.lang.NullPointerException
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthType$1(TreeUnpickler.scala:362)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readType(TreeUnpickler.scala:396)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readType$19(TreeUnpickler.scala:391)
[error] scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:454)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readSimpleType$1(TreeUnpickler.scala:391)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readType(TreeUnpickler.scala:396)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readType$9(TreeUnpickler.scala:340)
[error] scala.tools.tasty.TastyReader.$anonfun$until$1(TastyReader.scala:135)
[error] scala.tools.tasty.TastyReader.doUntil(TastyReader.scala:140)
[error] scala.tools.tasty.TastyReader.until(TastyReader.scala:135)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthType$1(TreeUnpickler.scala:340)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readType(TreeUnpickler.scala:396)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readType$9(TreeUnpickler.scala:340)
[error] scala.tools.tasty.TastyReader.$anonfun$until$1(TastyReader.scala:135)
[error] scala.tools.tasty.TastyReader.doUntil(TastyReader.scala:140)
[error] scala.tools.tasty.TastyReader.until(TastyReader.scala:135)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthType$1(TreeUnpickler.scala:340)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readType(TreeUnpickler.scala:396)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthType$1(TreeUnpickler.scala:345)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readType(TreeUnpickler.scala:396)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readParamTypes(TreeUnpickler.scala:227)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readType$4(TreeUnpickler.scala:310)
[error] scala.tools.nsc.tasty.bridge.TypeOps$HKTypeLambda.<init>(TypeOps.scala:644)
[error] scala.tools.nsc.tasty.bridge.TypeOps$HKTypeLambda$.factory(TypeOps.scala:559)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readType$2(TreeUnpickler.scala:308)
[error] scala.collection.mutable.HashMap.getOrElse(HashMap.scala:436)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthType$1(TreeUnpickler.scala:303)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readType(TreeUnpickler.scala:396)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readSimpleTerm$1(TreeUnpickler.scala:1000)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1109)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTpt(TreeUnpickler.scala:1121)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readSimpleTerm$1(TreeUnpickler.scala:1008)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1109)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1022)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1109)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1043)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1109)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1036)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1109)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readTemplate$6(TreeUnpickler.scala:884)
[error] scala.tools.tasty.TastyReader.collectWhile(TastyReader.scala:151)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.traverseParents$1(TreeUnpickler.scala:883)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.traverseTemplate$1(TreeUnpickler.scala:919)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readTemplate$11(TreeUnpickler.scala:927)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.$anonfun$readTemplate$11$adapted(TreeUnpickler.scala:927)
[error] scala.tools.nsc.tasty.bridge.ContextOps.inInnerScopeContext(ContextOps.scala:100)
[error] scala.tools.nsc.tasty.bridge.ContextOps.inInnerScopeContext$(ContextOps.scala:98)
[error] scala.tools.nsc.tasty.TastyUniverse.inInnerScopeContext(TastyUniverse.scala:20)
[error] scala.tools.nsc.tasty.bridge.ContextOps.$anonfun$inIndexScopedStatsContext$1(ContextOps.scala:107)
[error] scala.tools.nsc.tasty.bridge.ContextOps.$anonfun$inIndexScopedStatsContext$1$adapted(ContextOps.scala:107)
[error] scala.tools.nsc.tasty.bridge.ContextOps.inIndexStatsContext(ContextOps.scala:93)
[error] scala.tools.nsc.tasty.bridge.ContextOps.inIndexStatsContext$(ContextOps.scala:91)
[error] scala.tools.nsc.tasty.TastyUniverse.inIndexStatsContext(TastyUniverse.scala:20)
[error] scala.tools.nsc.tasty.bridge.ContextOps.inIndexScopedStatsContext(ContextOps.scala:107)
[error] scala.tools.nsc.tasty.bridge.ContextOps.inIndexScopedStatsContext$(ContextOps.scala:106)
[error] scala.tools.nsc.tasty.TastyUniverse.inIndexScopedStatsContext(TastyUniverse.scala:20)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readTemplate(TreeUnpickler.scala:927)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.TypeDef$1(TreeUnpickler.scala:806)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.initialize$1(TreeUnpickler.scala:841)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.initializeMember(TreeUnpickler.scala:847)
[error] scala.tools.nsc.tasty.TreeUnpickler$TreeReader.readIndexedMember(TreeUnpickler.scala:716)
[error] scala.tools.nsc.tasty.TreeUnpickler$Completer.$anonfun$computeInfo$1(TreeUnpickler.scala:106)
[error] scala.tools.nsc.tasty.bridge.ContextOps$Context.withPhaseNoLater(ContextOps.scala:505)
[error] scala.tools.nsc.tasty.TreeUnpickler$Completer.computeInfo(TreeUnpickler.scala:105)
[error] scala.tools.nsc.tasty.bridge.TypeOps$TastyCompleter.complete(TypeOps.scala:411)
[error] scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1561)
[error] scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1533)
[error] scala.reflect.internal.Types$TypeRef.baseClasses(Types.scala:2617)
[error] scala.reflect.internal.Types$SimpleTypeProxy.baseClasses(Types.scala:171)
[error] scala.reflect.internal.Types$SimpleTypeProxy.baseClasses$(Types.scala:171)
[error] scala.reflect.internal.Types$SingletonType.baseClasses(Types.scala:1111)
[error] scala.reflect.internal.tpe.FindMembers$FindMemberBase.init(FindMembers.scala:37)
[error] scala.reflect.internal.tpe.FindMembers$FindMember.init(FindMembers.scala:258)
[error] scala.reflect.internal.Types$Type.$anonfun$findMember$1(Types.scala:1043)
[error] scala.reflect.internal.Types$Type.findMemberInternal$1(Types.scala:1042)
[error] scala.reflect.internal.Types$Type.findMember(Types.scala:1047)
[error] scala.reflect.internal.Types$Type.memberBasedOnName(Types.scala:673)
[error] scala.reflect.internal.Types$Type.nonLocalMember(Types.scala:664)
[error] scala.tools.nsc.typechecker.Typers$Typer.member(Typers.scala:669)
[error] scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$57(Typers.scala:5259)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedSelect$1(Typers.scala:5259)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedSelectOrSuperCall$1(Typers.scala:5411)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5986)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6041)
[error] scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$41(Typers.scala:5074)
[error] scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:698)
[error] scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:5076)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:5104)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5985)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6041)
[error] scala.tools.nsc.typechecker.Typers$Typer.computeType(Typers.scala:6130)
[error] scala.tools.nsc.typechecker.Namers$Namer.assignTypeToTree(Namers.scala:1138)
[error] scala.tools.nsc.typechecker.Namers$Namer.methodSig(Namers.scala:1460)
[error] scala.tools.nsc.typechecker.Namers$Namer.memberSig(Namers.scala:1934)
[error] scala.tools.nsc.typechecker.Namers$Namer.typeSig(Namers.scala:1885)
[error] scala.tools.nsc.typechecker.Namers$Namer$MonoTypeCompleter.completeImpl(Namers.scala:885)
[error] scala.tools.nsc.typechecker.Namers$LockingTypeCompleter.complete(Namers.scala:2083)
[error] scala.tools.nsc.typechecker.Namers$LockingTypeCompleter.complete$(Namers.scala:2081)
[error] scala.tools.nsc.typechecker.Namers$TypeCompleterBase.complete(Namers.scala:2076)
[error] scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1561)
[error] scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1533)
[error] scala.reflect.internal.Symbols$Symbol.initialize(Symbols.scala:1722)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5625)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6041)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:6119)
[error] scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$8(Typers.scala:3410)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3410)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:2064)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:1940)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5952)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6041)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:6119)
[error] scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$8(Typers.scala:3410)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3410)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5634)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5954)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6041)
[error] scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.apply(Analyzer.scala:112)
[error] scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:454)
[error] scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.run(Analyzer.scala:101)
[error] scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1519)
[error] scala.tools.nsc.Global$Run.compileUnits(Global.scala:1503)
[error] scala.tools.nsc.Global$Run.compileSources(Global.scala:1495)
[error] scala.tools.nsc.Global$Run.compileFiles(Global.scala:1609)
[error] xsbt.CachedCompiler0.run(CompilerBridge.scala:163)
[error] xsbt.CachedCompiler0.run(CompilerBridge.scala:134)
[error] xsbt.CompilerBridge.run(CompilerBridge.scala:39)
[error] sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:186)
[error] scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:241)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:176)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:157)
[error] sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:157)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:204)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:528)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:528)
[error] sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:174)
[error] sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:172)
[error] sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:457)
[error] sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
[error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
[error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
[error] sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:261)
[error] sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:412)
[error] sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:499)
[error] sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:399)
[error] sbt.internal.inc.Incremental$.apply(Incremental.scala:166)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:528)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:482)
[error] sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:420)
[error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
[error] sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2362)
[error] sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2319)
[error] sbt.internal.io.Retry$.apply(Retry.scala:46)
[error] sbt.internal.io.Retry$.apply(Retry.scala:28)
[error] sbt.internal.io.Retry$.apply(Retry.scala:23)
[error] sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:31)
[error] sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2315)
[error] scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] sbt.Execute.work(Execute.scala:291)
[error] sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] java.base/java.lang.Thread.run(Thread.java:834)

Expected behaviour: it tells me about the compile errors in my own code
Actual behaviour: the compiler breaks with an NPE in the tasty reader.

[griggt]

Minimized to:

build.sbt

lazy val api = (project in file("api"))
  .settings(
    scalaVersion := "3.0.0",
  )

lazy val client = (project in file("client"))
  .dependsOn(api)
  .settings(
    scalaVersion := "2.13.6",
    scalacOptions += "-Ytasty-reader",
  )

api/Lib.scala

trait HasId[+K] {
  def id: K
}

trait Id[+T, K] {
  def id: K
}

case class UserId(id: String) extends Id[User, String]
case class User(id: UserId) extends HasId[UserId]

class Bar[A <: HasId[Id[A, String]]](bar: A)
class Foo extends Bar(User(???))

client/Test.scala

object Test {
  def foo = new Foo
}

[wbillingsley]

There's a few JitPack snapshot dependencies in that repo, so this comment is just to give which commits (can sub the hash in where it says "master-SNAPSHOT", "v0.11-SNAPSHOT" etc) they are according to jitpack at the moment. As otherwise, if it takes a while to look into this and those snapshots move on, compile errors in the dependent modules could stop it reaching the play project's compile stage.

veautiful: 6c2ac65049
veautiful-templates: 6c2ac65049
handy: d267b0c485

Tom Grigg on gitter kindly mentions that he may have a more minimal example that can reproduce it however.

[bishabosha]

Thanks for the report! and for the minimisation

it looks like an issue of not being lazy enough in the tasty reader, so parameters are referenced before their symbol is created