Upgrade Scala 3.x versions

[pjfanning]

Change to

• Scala 2.3.10 due to CVE - https://www.cvedetails.com/cve/CVE-2022-36944/
• Scala 3.2.0 -- Scala team recommend using latest Scala 3 version - https://www.scala-lang.org/blog/2022/08/17/long-term-compatibility-plans.html
• Scala 3.3.0 - the first LTS version of Scala 3 should be out soon

[alexandru]

Bumping Scala 3's minor version isn't necessarily a good thing, some care is needed.

If a project is using Scala 3.1.3, bumping the Pekko dependency to a new version that uses Scala 3.2.0, also forces the project on Scala 3.2.0 — backwards compatibility is provided, but not forward compatibility. In that sense, Scala 3 upgrades behave like libraries. This is unlike upgrading from Scala 2.3.9 to Scala 2.3.10.

And care is needed because a compiler is unlike a library, as it may have bugs that don't manifest in the Pekko library's codebase, but that impacts downstream projects.

I think right now an upgrade to Scala 3.2.0 is totally fine, given the state of the Scala 3 adoption, and that of Pekko, but for the future, it's good to keep in mind that such upgrades basically force the new Scala version on downstream users, and that may not be good for Scala versions that have issues.

[pjfanning]

@alexandru thanks for the comments.

Picking out comments from the article linked above, just in case.

You shouldn’t be afraid of updating it. Bumping the patch version (e.g. 3.1.2 -> 3.1.3) is a no-brainer. Bumping the minor version (e.g. 3.1.3 -> 3.2.0) requires the same considerations as changing the minor version of any other dependency, yet is generally advisable.

So, upgrading to 3.2.0 is not zero risk but it pretty low on the scale of these things.

I'm not going to do anything with this issue yet and it is probably best to discuss this item on the dev mailing list before proceeding to the PR stage.

[mdedetrich]

I think the CVE may work for a Pekko release of 1.0.x pending on what we ultimately decide goes into 1.0.x (since its a CVE) but 3.2.0 makes sense for 1.1.x

[alexandru]

@pjfanning

upgrading to 3.2.0 is not zero risk but it pretty low on the scale of these things.

I kind of disagree with their blog post, as it's not low risk, because new compiler versions can introduce source-level regressions. The difference between a library and the compiler, is that libraries are dynamically linked, at runtime. For this reason, a tranzitive dependency cannot break your source code, but compiler update can.

For example, here's an issue that broke my code when upgrading from 3.0.x to 3.1.2, and it was close to be considered a feature: scala/scala3#15160

Downstream users may be able to fix such instances, but if an upgrade breaks source compatibility, then in my view that's a major version, not a minor one (according to semver).

---

Again, 3.2.0 is fine for now, all I'm saying is that we can't be very relaxed about upgrading these versions. I would feel better if we let new versions stabilize a little.

And I also don't think we need further discussions, it's a simple update IMO, and for now the Scala 3 adoption of Akka is probably very low anyway, so might as well update it now, rather than later. But it's something to keep in mind.

[nvollmar]

I would argue against using 3.2.0 as it forces downstream users/projects to also upgrade to 3.2.0.
Last time I checked, Scala 3.1 couldn't read TASTy files of 3.2 and failed to compile using that library.

[MateuszKubuszok]

For now Scala 3 adoption is rather low (among other reasons - Akka HTTP is not released for any version of Scala 3, and apparently it uses Scala 2 macros), so these updates wouldn't affect many people. Surely none that use Akka HTTP and would like to migrate to Pekko.

According to the blog post - by the end of the year the compiler team want to release the first LTS version of Scala 3 (3.3). So the current situation is rather a temporary flux period before LTS kicks in. If 3.3 would be supported for at least 2 years as claimed, it would make sense to release Pekko for 3.3, stay on it until next LTS is adopted and then move on to next LTS.

IMHO, moving fast for now make sense. For many libraries blocker to update are macros and I notices that some important features stopped being @experimental only in 3.2.0, which would block importing certain functionalities before this version. And these libraries might slow down adoption for whole community, so I would expect for it to take off around the time that the compiler team want to release LTS, or even later.

Once LTS program goes live this whole issue with upgrading should be solved. Pekko can just use the current LTS and maybe wait a bit when new LTS is released. That should give community enough time to bump their versions.

[nvollmar]

Many things like IDE support are still very lacky for Scala 3, but we also don’t gain much by going to the newest Scala version. As long as Pekko supports Scala 2.13, we’re very limited in what we can use without having to resort to splitting source files by scala version.

[pjfanning]

Thanks @MateuszKubuszok for the detailed status update. akka-http is not yet released with Scala 3 support but the core akka libs do support Scala 3.

After I do some experimentation, I might look at adding Pekko to the Dotty Community Build.

[mdedetrich]

According to the blog post - by the end of the year the compiler team want to release the first LTS version of Scala 3 (3.3). So the current situation is rather a temporary flux period before LTS kicks in. If 3.3 would be supported for at least 2 years as claimed, it would make sense to release Pekko for 3.3, stay on it until next LTS is adopted and then move on to next LTS.

If this is the case then at least to me it seems like doing the first release with Scala 3.3 is the best option? In terms of both it being an LTS and maybe in terms of timing (i.e. by the time pekko-http is ready to be released). I wasn't aware that Scala 3 basically has different stability guarantees, it sounds like there is a risk that we may end up having to having a matrix if we don't pick a good Scala3 version and a lot of people end up depending on it?

[nvollmar]

Scala 3.1.3 would be the safest bet for now. Anyone on Scala 3.1 or 3.2 could use it. As long as we cross compile for Scala 2.13 we can't use many Scala 3 specific features anyway.

[MateuszKubuszok]

I wasn't aware that Scala 3 basically has different stability guarantees, it sounds like there is a risk that we may end up having to having a matrix if we don't pick a good Scala3 version and a lot of people end up depending on it?

Scala 2 used epoch.major.minor version convention. Scala 3 has major.minor.patch.

So there is no 3.0/3.1/3.2/etc cross-compilation - the assumption is that:

• you can compile against the same minor version with backward- and forward-compatibility: 3.1.3 dependency against 3.1.0 code, 3.0.0 dependency against 3.0.1 code, etc
• within the same major version you always have backward-compatibility: 3.1.3 dependency can be used in 3.1.3 project, but also 3.2.0 project and in future against 3.3.0 project

It means that you should use the lowest version that you can get away with to not force people to update their build unnecessarily. If it would be e.g. 3.1.3 then there is a case to use 3.1.3 for you project and not update it.

At the same time, before 3.3 there is no LTS version planned. Meaning that if there are bugs, regressions, security vulnerabilities - they won't be backported to versions prior to 3.3, once 3.3 is released. If these issues appear in code compiled with e.g. 3.1.3 when 3.3 will be already released, there won't be any new release for 3.1 (or 3.2) fixing them. It all mirrors the JVM approach where you have LTS for Java 11 and Java 17 but intermediate versions are EOLed rather fast, and while you are encouraged to test with them, you are not given any long term support guarantees for them.

So while thing will be much clearer after the release of 3.3, right now we are living in the period of versions that can be used for any future development but will be supported for at best next few months (post suggest 3.3 release at "late 2022").

Akka core might get away with 3.1.3 or earlier.

Akka HTTP from what I understand relies on parboiled2 for its macros - 2.4.1 was compiled with 3.2.1 and 2.4.0 was compiled with 3.1.1, so it might require some investigation.

[alexandru]

@nvollmar

Many things like IDE support are still very lacky for Scala 3, but we also don’t gain much by going to the newest Scala version. As long as Pekko supports Scala 2.13, we’re very limited in what we can use without having to resort to splitting source files by scala version.

Just to set some expectations, in my experience, it is impossible to target both Scala 2.13 and Scala 3 without resorting to source files specific per version. And the setup can get quite complicated. Which is fine, at least we can. Only the simplest of libraries can get away with only shared code that compiles in both.

@MateuszKubuszok

Akka HTTP from what I understand relies on parboiled2 for its macros - 2.4.1 was compiled with 3.2.1 and 2.4.0 was compiled with 3.1.1, so it might require some investigation.

I was just thinking of this — we might not have a choice, but update, alongside our dependencies. For example, akka-persistence depends on jackson-module-scala, and the latest version also depends on Scala 3.2.0.

I think this idea is good — try to depend on the 3.3 LTS, when it gets released, and until then, depend on intermediary versions 👍

[jrudolph]

On Fri, Nov 4, 2022 at 9:21 AM Alexandru Nedelcu ***@***.***> wrote: @MateuszKubuszok <https://github.com/MateuszKubuszok> Akka HTTP from what I understand relies on parboiled2 for its macros - 2.4.1 was compiled with 3.2.1 and 2.4.0 was compiled with 3.1.1, so it might require some investigation. I was just thinking of this — we might not have a choice, but update, alongside our dependencies. For example, akka-persistence depends on jackson-module-scala <https://github.com/FasterXML/jackson-module-scala>, and the latest version also depends on Scala 3.2.0. I think this idea is good — try to depend on the 3.3 LTS, when it gets released, and until then, depend on intermediary versions 👍

This is a good point. Another point is that Akka's Scala 3 support has so far been experimental and incomplete so expectations on compatibility are low. Until we make the first release, I think we should grant ourselves the flexibility of tracking the latest version of Scala 3. Not least, because Scala 3 has been somewhat unstable and we hit many regressions during the akka-http migration so we might have to take whatever works... Incidentally, Akka HTTP right now contains a copy of parboiled2 in its sources (with some slight additions). This decision was originally made on the grounds that Akka should not have many dependencies on Scala libraries, so it can be released quickly after a new (incompatible) Scala versions comes up. This point does not seem as important as before, so I'd suggest to move upstream parboiled2 when possible. I created apache/pekko-http#1 to track this.

…

Message ID: ***@***.***>