GradleBuild is not compatible with composite builds

[rupebac]

I know the reasons why you used this GradleBuild:

#62

But we had a great surprise when we tried to release our recently new stuff using composite builds. GradleBuild is not supporting them:

gradle/gradle#8246

This is a huge limitation. Composite Builds are receiving everytime more attention.

We have managed to workaround it by splitting it in 3 phases and calling it from different build cycles from the pipeline:

phase 1:

task releaseSym_1 {
    dependsOn "createScmAdapter"
    dependsOn "initScmAdapter"
    dependsOn "checkCommitNeeded"
    dependsOn "checkUpdateNeeded"
    dependsOn "checkoutMergeToReleaseBranch"
    dependsOn "unSnapshotVersion"
    dependsOn "confirmReleaseVersion"
    dependsOn "checkSnapshotDependencies"
}

phase 2:
then we call our build stuff

phase 3:

task releaseSym_2 {
    dependsOn "createScmAdapter"
    dependsOn "initScmAdapter"
    dependsOn "preTagCommit"
    dependsOn "createReleaseTag"
    dependsOn "checkoutMergeFromReleaseBranch"
    dependsOn "updateVersion"
    dependsOn "commitNewVersion"
}

I also have to rework the "mustRunAfter" to guarantee the order.

In the end it is kind of working, but it is not nice at all. Would you be willing to accept Pull request and work on it together?

[Hillkorn]

Why do you try to use composite builds for this and not a multi project setup? I think composite builds are not really made for this as they just shall help to develop in multiple projects. In multi project setups you explicitly specify the dependency in a project but for composite builds you get a drop in replacement of the referenced projects and their sub projects.
How do you handle the difference of the declared dependencies and the referenced project? Because you might specify projectX with version 1.0.0 as a compile dependency but the composite build build brings in version 2.0.0 and if you don't update declaration the build is not really reproducible except you look into the pom.xml (I think it contains the replaced version) to know which version was really used at build.
We actually have a check that doesn't allow releases with composite build setups because of that.

[rupebac]

@Hillkorn sorry I recover this topic now, just forgot about it.
I think I get what you mean, while I would agree with you a few years ago, things have evolve in a different direction.
The problem you describe is not intrinsic with composite builds, it is already happening in multi project setups. You may be depending on junit:4.2, and that having a non strict version requirement on, commons-lang for instance. something like ":commons-lang3:latest.release". Am I understanding you correctly?

This is why you have dependency locking in gradle:
https://docs.gradle.org/current/userguide/dependency_locking.html

And also, keep in mind the new concept of "platforms" in Gradle. A composite build can share one platform (which we do), so all version requirements are actually "shared", so one project included in the composite build cannot violate any of our version rules.

[Vampire]

This is really a problem.
I also agree that times where composite build were just for drop-in replacement are long gone.
Composite builds are used for a better buildSrc replacement, they are used for another level of structuring builds, they are used if you use the built-in source dependencies feature, they are used if you use the includegit-gradle-plugin, ....

The last one in the list is for example my current situation.
I'm developing three private projects, one is a base for the other too, let's call them common, A, and B.
A and B use the includegit-gradle-plugin to get a specific tag of common automatically cloned and included, as there is no private repository where these artifacts are going to be published.

So please fix this somehow.

[Vampire]

Using the tooling api to kick off the build instead of the GradleBuild task type works fine, so that should probably be used instead, it is also what IDEs use, so it is highly likely that it will be compatible with future changes.

For example:

tasks.register("runBuildTasks") {
    doLast {
        GradleConnector
            .newConnector()
            .forProjectDirectory(layout.projectDirectory.asFile)
            .connect()
            .use { projectConnection ->
                projectConnection
                    .newBuild()
                    .forTasks("build")
                    .setStandardInput(System.`in`)
                    .setStandardOutput(System.out)
                    .setStandardError(System.err)
                    .run()
            }
    }
}

That's in Kotlin DSL, but you should be able to easily transform it to Groovy too.

[Vampire]

Yep, this works as consumer-side work-around, replacing the GradleBuild task action by a tooling API call:

configure(listOf(tasks.release, tasks.runBuildTasks)) {
    configure {
        actions.clear()
        doLast {
            GradleConnector
                .newConnector()
                .forProjectDirectory(layout.projectDirectory.asFile)
                .connect()
                .use { projectConnection ->
                    val buildLauncher = projectConnection
                        .newBuild()
                        .forTasks(*tasks.toTypedArray())
                        .setStandardInput(System.`in`)
                        .setStandardOutput(System.out)
                        .setStandardError(System.err)
                    gradle.startParameter.excludedTaskNames.forEach {
                        buildLauncher.addArguments("-x", it)
                    }
                    buildLauncher.run()
                }
        }
    }
}

So it should also work fine if you do it in the plugin right away.
You probably need to add a bit more gradle.startParameter to commandline argument translations like -s and -S and so on.

[Hillkorn]

@Vampire Your suggestion would also help to fix #346 I think. We can get the result inside the task and do the cleanup on failure.

[Vampire]

@Vampire Your suggestion would also help to fix #346 I think. We can get the result inside the task and do the cleanup on failure.

The afterTask part probably, yes.

[Hillkorn]

But this would remove the need for the listener as we only listen for failures of the release task to run scmAdapter.revert() if configured.

[Vampire]

This issue has actually one more implication.
From Gradle 7 on the type-safe accessors for Kotlin DSL precompiled script plugins cannot be generated anymore and from Gradle 8 on this results in a hard failure of the build.
Here the according Gradle issue: gradle/gradle#23747
If you would not use GradleBuild, but the tooling API as suggested, I think this should also work properly again.

[Vampire]

Or if you would not eagerly create the tasks, according to that issue, but not using GradleBuild would be better of course.