Bug: Spotless gradle plugin execution fails with Gradle 8.0

[dejan2609]

Note: related to apache/kafka#13205

Summary of the problem: 'spotlessScalaCheck' gradle task works as expected in isolation, but fails when combined with other Gradle tasks.

Gradle version: 8.0
Spotless Gradle plugin version: 6.15.0
Java version: 11
OS and version: Ubuntu 22.04

Reproducer: https://github.com/dejan2609/kafka/tree/gradle-8-java-11-spotlessPluginGradle-6.15.0

Step-by-step:

• git clone https://github.com/dejan2609/kafka.git
• cd kafka
• git checkout gradle-8-java-11-spotlessPluginGradle-6.15.0
• execute ./gradlew clean spotlessScalaCheck ➡️ build works fine ✅, but
• execute same taks combined with other tasks (this long command is used on CI server, see Jenkinsfile):
  ./gradlew -PscalaVersion=2.13 clean compileJava compileScala compileTestJava compileTestScala spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat --profile --continue -PxmlSpotBugsReport=true -PkeepAliveMode="session"
• and build breaks ❌ (see below for more details).

dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$ date
Wed 15 Feb 15:39:17 CET 2023
dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$ java -version
openjdk version "11.0.17" 2022-10-18
OpenJDK Runtime Environment (build 11.0.17+8-post-Ubuntu-1ubuntu222.04)
OpenJDK 64-Bit Server VM (build 11.0.17+8-post-Ubuntu-1ubuntu222.04, mixed mode, sharing)
dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$ git log -3 --oneline
4c366b4e23 (HEAD -> gradle-8-java-11-spotlessPluginGradle-6.15.0, origin/gradle-8-java-11-spotlessPluginGradle-6.15.0) Spotless Gradle plugin version is bumped from 6.13.0 to 6.15.0
0e82675469 (origin/gradle-8-test, origin/gradle-8, gradle-8-test, gradle-8) KAFKA-14680: gradle version upgrade 7 -->> 8
b9754747d6 (upstream/trunk, origin/trunk, origin/HEAD, trunk) KAFKA-14653: Use raw properties instead of post-resolution properties for MirrorMaker connectors(#13163)
dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$ ./gradlew clean spotlessScalaCheck

> Configure project :
Starting build with version 3.5.0-SNAPSHOT (commit id 4c366b4e) using Gradle 8.0, Java 11 and Scala 2.13.10
Build properties: maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0

BUILD SUCCESSFUL in 5s
51 actionable tasks: 4 executed, 47 up-to-date
dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$ ./gradlew -PscalaVersion=2.13 clean compileJava compileScala compileTestJava compileTestScala spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat --profile --continue -PxmlSpotBugsReport=true -PkeepAliveMode="session"

> Configure project :
Starting build with version 3.5.0-SNAPSHOT (commit id 4c366b4e) using Gradle 8.0, Java 11 and Scala 2.13.10
Build properties: maxParallelForks=8, maxScalacThreads=8, maxTestRetries=0

> Task :spotlessScala FAILED

> Task :core:compileScala
Unexpected javac output: warning: [options] bootstrap class path not set in conjunction with -source 8
1 warning.

> Task :core:compileTestScala
Unexpected javac output: warning: [options] bootstrap class path not set in conjunction with -source 8
1 warning.

FAILURE: Build failed with an exception.

* What went wrong:
Some problems were found with the configuration of task ':spotlessScala' (type 'SpotlessTaskImpl').
  - Gradle detected a problem with the following location: '/home/dejan/IdeaProjects/kafka'.
   
    Reason: Task ':spotlessScala' uses this output of task ':streams:processResources' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
   
    Possible solutions:
      1. Declare task ':streams:processResources' as an input of ':spotlessScala'.
      2. Declare an explicit dependency on ':streams:processResources' from ':spotlessScala' using Task#dependsOn.
      3. Declare an explicit dependency on ':streams:processResources' from ':spotlessScala' using Task#mustRunAfter.
   
    Please refer to https://docs.gradle.org/8.0/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/home/dejan/IdeaProjects/kafka'.
   
    Reason: Task ':spotlessScala' uses this output of task ':streams:processTestResources' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
   
    Possible solutions:
      1. Declare task ':streams:processTestResources' as an input of ':spotlessScala'.
      2. Declare an explicit dependency on ':streams:processTestResources' from ':spotlessScala' using Task#dependsOn.
      3. Declare an explicit dependency on ':streams:processTestResources' from ':spotlessScala' using Task#mustRunAfter.
   
    Please refer to https://docs.gradle.org/8.0/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/home/dejan/IdeaProjects/kafka'.
   
    Reason: Task ':spotlessScala' uses this output of task ':streams:upgrade-system-tests-0100:compileTestJava' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
   
    Possible solutions:
      1. Declare task ':streams:upgrade-system-tests-0100:compileTestJava' as an input of ':spotlessScala'.
      2. Declare an explicit dependency on ':streams:upgrade-system-tests-0100:compileTestJava' from ':spotlessScala' using Task#dependsOn.
      3. Declare an explicit dependency on ':streams:upgrade-system-tests-0100:compileTestJava' from ':spotlessScala' using Task#mustRunAfter.
   
    Please refer to https://docs.gradle.org/8.0/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/home/dejan/IdeaProjects/kafka'.
   
    Reason: Task ':spotlessScala' uses this output of task ':streams:upgrade-system-tests-0101:compileTestJava' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
   
    Possible solutions:
      1. Declare task ':streams:upgrade-system-tests-0101:compileTestJava' as an input of ':spotlessScala'.
      2. Declare an explicit dependency on ':streams:upgrade-system-tests-0101:compileTestJava' from ':spotlessScala' using Task#dependsOn.
      3. Declare an explicit dependency on ':streams:upgrade-system-tests-0101:compileTestJava' from ':spotlessScala' using Task#mustRunAfter.
   
    Please refer to https://docs.gradle.org/8.0/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/home/dejan/IdeaProjects/kafka'.
   
    Reason: Task ':spotlessScala' uses this output of task ':streams:upgrade-system-tests-0102:compileTestJava' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
   
    Possible solutions:
      1. Declare task ':streams:upgrade-system-tests-0102:compileTestJava' as an input of ':spotlessScala'.
      2. Declare an explicit dependency on ':streams:upgrade-system-tests-0102:compileTestJava' from ':spotlessScala' using Task#dependsOn.
      3. Declare an explicit dependency on ':streams:upgrade-system-tests-0102:compileTestJava' from ':spotlessScala' using Task#mustRunAfter.
   
    Please refer to https://docs.gradle.org/8.0/userguide/validation_problems.html#implicit_dependency for more details about this problem.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org/

BUILD FAILED in 8m 7s
251 actionable tasks: 196 executed, 55 up-to-date

See the profiling report at: file:///home/dejan/IdeaProjects/kafka/build/reports/profile/profile-2023-02-15-15-40-14.html
A fine-grained performance profile is available: use the --scan option.
dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$ date
Wed 15 Feb 15:48:53 CET 2023
dejan@dejan-HP-ProBook-450-G7:~/IdeaProjects/kafka$

[cocorossello]

I think you should move your spotless scala configuration below, where scala plugin is applied. The problem is that where it is right now it doesn't "understand" about scala, that's why it fails.

This patch worked for me
apply_spotless_plugin_where_scala_is_applied.patch

[dejan2609]

Thank you @cocorossello, your workaround does the trick !

This is also a valuable info for Spotless people (it will help them to provide permanent solution).

[ikovalyov]

I see same issue in kotlin multiplatform project:

Some problems were found with the configuration of task ':spotlessKotlin' (type 'SpotlessTaskImpl').
  - Gradle detected a problem with the following location: '/Users/illia.kovalov/kotlin-blogpost-engine'.
    
    Reason: Task ':spotlessKotlin' uses this output of task ':compileKotlinJs' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':compileKotlinJs' as an input of ':spotlessKotlin'.
      2. Declare an explicit dependency on ':compileKotlinJs' from ':spotlessKotlin' using Task#dependsOn.
      3. Declare an explicit dependency on ':compileKotlinJs' from ':spotlessKotlin' using Task#mustRunAfter.
    
    Please refer to https://docs.gradle.org/8.0.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/Users/illia.kovalov/kotlin-blogpost-engine'.
    
    Reason: Task ':spotlessKotlin' uses this output of task ':compileKotlinJvm' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':compileKotlinJvm' as an input of ':spotlessKotlin'.
      2. Declare an explicit dependency on ':compileKotlinJvm' from ':spotlessKotlin' using Task#dependsOn.
      3. Declare an explicit dependency on ':compileKotlinJvm' from ':spotlessKotlin' using Task#mustRunAfter.
    
    Please refer to https://docs.gradle.org/8.0.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/Users/illia.kovalov/kotlin-blogpost-engine'.
    
    Reason: Task ':spotlessKotlin' uses this output of task ':compileTestKotlinJvm' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':compileTestKotlinJvm' as an input of ':spotlessKotlin'.
      2. Declare an explicit dependency on ':compileTestKotlinJvm' from ':spotlessKotlin' using Task#dependsOn.
      3. Declare an explicit dependency on ':compileTestKotlinJvm' from ':spotlessKotlin' using Task#mustRunAfter.
    
    Please refer to https://docs.gradle.org/8.0.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/Users/illia.kovalov/kotlin-blogpost-engine'.
    
    Reason: Task ':spotlessKotlin' uses this output of task ':compileCommonMainKotlinMetadata' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':compileCommonMainKotlinMetadata' as an input of ':spotlessKotlin'.
      2. Declare an explicit dependency on ':compileCommonMainKotlinMetadata' from ':spotlessKotlin' using Task#dependsOn.
      3. Declare an explicit dependency on ':compileCommonMainKotlinMetadata' from ':spotlessKotlin' using Task#mustRunAfter.
    
    Please refer to https://docs.gradle.org/8.0.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/Users/illia.kovalov/kotlin-blogpost-engine'.
    
    Reason: Task ':spotlessKotlin' uses this output of task ':compileProductionExecutableKotlinJs' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':compileProductionExecutableKotlinJs' as an input of ':spotlessKotlin'.
      2. Declare an explicit dependency on ':compileProductionExecutableKotlinJs' from ':spotlessKotlin' using Task#dependsOn.
      3. Declare an explicit dependency on ':compileProductionExecutableKotlinJs' from ':spotlessKotlin' using Task#mustRunAfter.

[chkpnt]

Having the same issue with spotlessJava for a task, in which I use spotlessApply as a finalizer:

task updateGeneratedCode {
    group BasePlugin.BUILD_GROUP
    description "Updates src/generated/"
    dependsOn(updateGeneratedCodeFromWsdls, updateGeneratedCodeFromXsds)
    finalizedBy(updateWsdlsAvailableOnClasspath, "spotlessApply")
}

Output:

Some problems were found with the configuration of task ':dynS:spotlessJava' (type 'SpotlessTaskImpl').
  - Gradle detected a problem with the following location: '...'.
    
    Reason: Task ':dynS:spotlessJava' uses this output of task ':dynS:updateGeneratedCodeFromXsds' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':dynS:updateGeneratedCodeFromXsds' as an input of ':dynS:spotlessJava'.
      2. Declare an explicit dependency on ':dynS:updateGeneratedCodeFromXsds' from ':dynS:spotlessJava' using Task#dependsOn.
      3. Declare an explicit dependency on ':dynS:updateGeneratedCodeFromXsds' from ':dynS:spotlessJava' using Task#mustRunAfter.
    
    Please refer to https://docs.gradle.org/8.1.1/userguide/validation_problems.html#implicit_dependency for more details about this problem.

Adding spotlessJava explicitily to finalizedBy fixes the issue.

(Using spotless gradle plugin v6.18.0.)

[jandresmerlyn]

Hello, I'm having the same issue in a kotlin project. Using finalizedBy("spotlessKotlin") explicitly as suggested by @chkpnt in this comment didn't work for me, because spotlessKotlin is not found as a gradle task. I'm wondering if this is because only spotlessCheck and spotlessApply are available during gradle evaluation. Is there a way to declare this dependency I'm not seeing? Thank you very much in advance.

Reason: Task ':spotlessKotlin' uses this output of task ':myProject:myTask' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':myProject:myTask' as an input of ':spotlessKotlin'.
      2. Declare an explicit dependency on ':myProject:myTask' from ':spotlessKotlin' using Task#dependsOn.
      3. Declare an explicit dependency on ':myProject:myTask' from ':spotlessKotlin' using Task#mustRunAfter.

When declaring the dependency explicitly, I'm getting this:

Could not determine the dependencies of task ':myProject:myTask'.
> Task with path 'spotlessKotlin' not found in project ':myProject'.

[koppor]

Similar for a simple spotless configuration

@@ -30,10 +30,11 @@ plugins {
     id 'com.adarshr.test-logger' version '4.0.0'
 
     id 'jacoco'
 
     id 'checkstyle'
+    id 'com.diffplug.spotless' version '6.22.0'
     id 'org.openrewrite.rewrite' version '6.4.0'
 
     id 'project-report'
 
     id 'idea'
@@ -520,10 +521,19 @@ jacocoTestReport {
 checkstyle {
     // will only run when called explicitly from the command line
     sourceSets = []
 }
 
+spotless {
+    java {
+        importOrder('java', '', 'javax', '', 'org.jabref', '', '\\#')
+        removeUnusedImports()
+
+        // palantirJavaFormat()
+    }
+}
+
 rewrite {
     activeRecipe(
         'org.jabref.config.rewrite.cleanup'
     )
     exclusion (