Incremental compilation always recompiles all

[andreak]

I always get this:

Pruning sources from previous analysis, due to incompatible CompileSetup.

and a full recompile.

My pom.xml is:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-compiler-plugin</artifactId>
	<version>3.8.0</version>
	<configuration>
		<source>${build.jdk.version}</source>
		<target>${build.jdk.version}</target>
		<encoding>UTF-8</encoding>
		<compilerArgs>
			<arg>--add-modules</arg><arg>ALL-SYSTEM</arg>
		</compilerArgs>
	</configuration>
</plugin>

<plugin>
	<groupId>net.alchim31.maven</groupId>
	<artifactId>scala-maven-plugin</artifactId>
	<version>${version.scala-maven-plugin}</version>
	<configuration>
		<compilerPlugins>
			<compilerPlugin>
				<groupId>com.artima.supersafe</groupId>
				<artifactId>supersafe_${version.scalalibs}</artifactId>
				<version>${version.supersafe}</version>
			</compilerPlugin>
		</compilerPlugins>
		<jvmArgs>
			<jvmArg>-Xms64m</jvmArg>
			<jvmArg>-Xss4m</jvmArg>
			<jvmArg>-Xmx2g</jvmArg>
		</jvmArgs>
		<args>
			<arg>-release</arg>
			<arg>${build.jdk.version}</arg>
		</args>
		<scalaVersion>${version.scalalibs}</scalaVersion>
		<recompileMode>incremental</recompileMode>
	</configuration>
</plugin>

version.scalalibs=2.12.7
build.jdk.version=10
version.supersafe=1.1.7
version.scala-maven-plugin=3.4.4

Any hints?

[davidB]

from gatling/gatling#3245

sbt misleadingly issue a WARN message every time it has to full compile, eg on initial run.

Sorry, but there are not support on incremental Mode which use an old zinc/sbt fragment (TBH, I planned to removed it).

[andreak]

No incremental mode?? That's very sad...

[davidB]

• I never use it (it was a contribution from typesafe)
• current mode generate some annoying bug
• nobody want to support it (require to upgrade to new version of zinc)
• some alternatives maven plugin (see README) provide more up-to-date intregration

I keep it for existing user.

[davidB]

see #208

[slandelle]

No incremental mode

IMHO, this feature is poorly named: it's not about incremental compilation, which is perfectly feasible with simple zinc as it can persist analysis between runs.
This feature is actually about nailgun: connecting to a compilation server process.

nobody want to support it (require to upgrade to new version of zinc)

As I successfully upgraded Gatling compiler module to Zinc 1 for Gatling 3, I gave a shot at trying to contribute a zinc upgrade a few months ago.
I gave up exactly because of this nailgun support:

• Zinc is utterly not documented. I was able to figure out how to upgrade to Zinc 1 for basic usage, but I have no idea what to do about
• I personally really don't need it (nor am convinced it's really helpful) so I can't waste days exploring those uncharted lands...

TBH, I planned to removed it

+1000
If you were to drop it, I'd happily contribute a Zinc upgrade.

some alternatives maven plugin (see README) provide more up-to-date intregration

Nah. This other plugin is not stable and there hasn't been activity there for 6 months (not blaming the author, just stating).

[davidB]

@slandelle,

TBH, I planned to removed it

I'll probably not drop zinc.

I'd happily contribute a Zinc upgrade.

Feel free to contribute (or not). TBH I waited from your feedback after release of zinc 1.0 ;-) when I saw some of your comments about it on other project, and I also noticed the lake of documentation and usable sample.

Nah. This other plugin is not stable and there hasn't been activity there for 6 months (not blaming the author, just stating).

scala-maven-plugin has no real activity since a long time, and often I didn't see issue notification. I'm surprise by the usage of this plugin in an ecosystem where sbt is the defacto build tool.

I used maven only for CI or clean build, (IDE for incremental). At work we use sbt for scala project, At Home, currently I mainly code in Rust.

[slandelle]

scala-maven-plugin has no real activity since a long time

It does the job very well for standard usage.

I'm surprise by the usage of this plugin in an ecosystem where sbt is the defacto build tool.

Well, for me, your plugin is irreplaceable. With Gatling, we have more Java users than Scala ones and we'll never want to force sbt complexity onto our users.

At work we use sbt for scala project

Ah, ah, I honestly thought you were doing Scala anymore :P

[jvican]

This feature is actually about nailgun: connecting to a compilation server process.

Bloop is a compilation server that we officially maintain at the Scala Center and that currently has support for Maven, Gradle, sbt and mill. The goal is to provide the fastest developer workflow for all Scala users, no matter what build tool they use. You can check the project here https://github.com/scalacenter/bloop and the docs https://scalacenter.github.io/bloop/. Bloop is the recommended tool to use to compile Scala in build tools.

I've never liked the idea that we're developing our own maven plugin, but it's been good for development as we've been able to move really fast and experiment. Internally, we depend on internals of scala-maven-plugin to be able to reuse as much code as possible from this project, as well as it's configuration (any project depending on this plugin should be able to add maven-bloop and have everything working). There are some rough edges (even though we have some happy maven users) so I would love help improving the maven-side of things in bloop to get a faster compilation for Maven users.

This being said, I would like to find a solution that allows us to replace the implementation of compile, test and run in Maven while reusing as much code from this plugin and the excellent community around it. I think it would be ideal if this plugin can depend on bloop directly and compile/test/run in an external server. This will simplify maintenance for the maintainers of this plugin (no more direct use of zinc and its intricate APIs) and make sure compilation is as fast as possible for Maven users.

I have some ideas to get to this ideal scenario, but I cannot lead this effort because I'm not a Maven user. However, I'm happy to help from the bloop side of the things (I'm working full-time on this project), all I would need is someone that can make some small changes in this plugin to fully integrate with bloop.

Would anyone in this thread be interested in that? Maybe @slandelle?

[andreak]

FWIW; It would be really nice to get incremental compilation of Scala-projects using Maven. We really depend on Maven at work and use many maven-plugins for which there really aren't any good Sbt-replacements. If Bloop can help with that I'm crossing my fingers....

[slandelle]

@jvican

Would anyone in this thread be interested in that? Maybe @slandelle?

Sorry, but I definitively will never have the cycles to work on this, even less become an active maintainer.

I feel like there's some confusion between 2 topics here.

1. Incremental compilation

This can be perfectly achieved with plain Zinc. There's no need for a compile server for his. If Bloop can also work in embedded mode and is better documented than Zinc, fine.

2. Remote server compilation

Lightbend contributed a nailgun support a few years ago, but now there's no one there to maintain it. IMHO, no one uses it. Making things worse: it's what stops upgrading Zinc and getting proper incremental compilation.
Such contributions are poisonous. Maintainers can't count on contributors to help maintaining.

I perfectly see how a compile server helps with long lived client processes such as IDEs and sbt in interactive mode. That's just my opinion, but I really fail to see how regular maven users would accept to install and start a side compile server service.

IMHO, as long as no one is willing to step up and maintain this plugin actively and on the long term, focus should be on core features, possibly reducing scope (dropping nailgun right away).

[jvican]

Agreed some comments here hint at a conflation of both incremental compilation and an external build server (remote server compilation is a misnomer).

Lightbend contributed a nailgun support a few years ago, but now there's no one there to maintain it.

You could see bloop as the successor of that old nailgun integration. Bloop runs in a nailgun server and supports nailgun clients.

That's just my opinion, but I really fail to see how regular maven users would accept to install and start a side compile server service.

If this process is completely automatic to the users, they won't care. For example, Gradle spawns a background process in the background and then the gradle client talks to it. Such mechanism is built-in, transparent to the user and helps improve the performance of the build. I have in mind doing something similar for Maven but only applied to Scala compilation.

IMHO, as long as no one is willing to step up and maintain this plugin actively and on the long term, focus should be on core features, possibly reducing scope (dropping nailgun right away).

The current maven-bloop plugin does all this plugin does but allows the use of bloop, using the newest Zinc (with the far better incremental compilation engine compared to 0.13.x).

[slandelle]

I have in mind doing something similar for Maven but only applied to Scala compilation.

That would be great. No idea if it's possible with maven though, all the more in multi-modules project.

The current maven-bloop plugin does all this plugin does but allows the use of bloop, using the newest Zinc (with the far better incremental compilation engine compared to 0.13.x).

My only concern (but I'm not the owner here) is to avoid one nighter style contribution as I suspect no one but you and the other people working on Bloop will be able to maintain it.

[jvican]

My only concern (but I'm not the owner here) is to avoid one nighter style contribution as I suspect no one but you and the other people working on Bloop will be able to maintain it.

I understand your concern. If this is done it won't be a "nighter style contribution" because:

1. The integration will be independent of compiler internals. You won't need to be an expert to maintain the integration.
2. Bloop will be maintained by the core Scala team as it's becoming more and more used in key infrastructure in our community.
3. Bloop versioning has a strict compatibility policy to ease migration on the build tools side.

A big problem of build tools in the Scala community, IMO, is that they have had to cope with integrating with fragile Scala compiler internals/test engines/Scala-specific logic for basic features. This is problematic because the Scala build tool plugins are often maintained by OSS plugin authors that don't have the time to work on migration such as the Zinc 0.13 -> Zinc 1.0 and we, at both the Scala Center and Lightbend, are only a few engineers overwhelmed with work.

I hope that removing the burden of integrating with compiler internals will be a positive move in our community, it will allow library authors to focus on more important things than migrating the Scala compilation engine and will allow us compiler maintainers to focus on improving compilation performance in a tool that every build tool can then reuse, hence shipping the best user experience to our Scala developers.

That would be great. No idea if it's possible with maven though, all the more in multi-modules project.

To elaborate on my ideas, I want to use the Build Server Protocol (https://github.com/scalacenter/bsp/) to offload compilation from Maven to Bloop. Bloop already implements the protocol and we provide a Java-only library to communicate via BSP, so all we need to do is to tie them together so that when the user does compile in maven it sends a request to the server and gets a response back. This can be done for multi-module builds too (I've been playing with this idea for quite a while in other integrations, such as the bloop-sbt integration, and it's been working nicely so far). Everything is more or less thought out, I just need the time to properly implement it. I will get back to the Maven integration when I'm done with all the other high-priority tasks I'm doing now.

[slandelle]

Thanks @jvican , looks promising!