forked from opensearch-project/OpenSearch
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Build local unreleased bwc versions more efficient for tests (7.x bac…
…kport) (#63188) * Wire local unreleased bwc versions more efficient for tests (#62473) For testing against the local distribution we already avoid the packaging/unpackaging cycle of es distributions when setting up test clusters. This PR adopts the usage of the expanded created distributions for unreleased bwc versions (versions that are checkout from a branch and build from source in the :distribution:bwc:minor / :distribution:bwc:bugfix). This makes the setup of bwc based cross version tests a bit faster by avoiding the unpackaging overhead. We still assemble both in the bwcBuild tasks atm which will be addressed in a later issue. This reworks the :distribution:bwc project: - Convert all the custom logic from build script logic (groovy) into gradle binary plugins (java) - Tried to make the bwc setup logic a bit more readable - Add basic functional test coverage for the bwc logic this PR tweaked. - Extracted a general internal BWC Git plugin out of the bwc setup plugin to improve maintenance and testability - Changed the InternalDistributionPlugin to resolve the extracted distro instead on relying on unpacking the distribution archive * Fix java8 incompatibility * Fix extension calculation for 6.8.* distribution
- Loading branch information
Showing
23 changed files
with
1,118 additions
and
379 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
51 changes: 51 additions & 0 deletions
51
...rc/integTest/groovy/org/elasticsearch/gradle/internal/InternalBwcGitPluginFuncTest.groovy
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| /* | ||
| * Licensed to Elasticsearch under one or more contributor | ||
| * license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright | ||
| * ownership. Elasticsearch licenses this file to you under | ||
| * the Apache License, Version 2.0 (the "License"); you may | ||
| * not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package org.elasticsearch.gradle.internal | ||
|
|
||
| import org.elasticsearch.gradle.fixtures.AbstractGradleFuncTest | ||
| import org.gradle.testkit.runner.TaskOutcome | ||
|
|
||
| class InternalBwcGitPluginFuncTest extends AbstractGradleFuncTest { | ||
|
|
||
| def setup() { | ||
| setupLocalGitRepo() | ||
| } | ||
|
|
||
| def "current repository can be cloned"() { | ||
| given: | ||
| internalBuild(); | ||
| buildFile << """ | ||
| import org.elasticsearch.gradle.Version; | ||
| apply plugin: org.elasticsearch.gradle.internal.InternalBwcGitPlugin | ||
| bwcGitConfig { | ||
| bwcVersion = project.provider { Version.fromString("7.10.0") } | ||
| bwcBranch = project.provider { "7.x" } | ||
| checkoutDir = project.provider{file("build/checkout")} | ||
| } | ||
| """ | ||
| when: | ||
| def result = gradleRunner("createClone", '--stacktrace').build() | ||
| then: | ||
| result.task(":createClone").outcome == TaskOutcome.SUCCESS | ||
| file("build/checkout/build.gradle").exists() | ||
| file("build/checkout/settings.gradle").exists() | ||
| } | ||
| } |
151 changes: 151 additions & 0 deletions
151
...roovy/org/elasticsearch/gradle/internal/InternalDistributionBwcSetupPluginFuncTest.groovy
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| /* | ||
| * Licensed to Elasticsearch under one or more contributor | ||
| * license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright | ||
| * ownership. Elasticsearch licenses this file to you under | ||
| * the Apache License, Version 2.0 (the "License"); you may | ||
| * not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package org.elasticsearch.gradle.internal | ||
|
|
||
| import org.apache.commons.io.FileUtils | ||
| import org.elasticsearch.gradle.fixtures.AbstractGradleFuncTest | ||
| import org.gradle.testkit.runner.TaskOutcome | ||
| import org.junit.Rule | ||
| import org.junit.rules.TemporaryFolder | ||
|
|
||
| class InternalDistributionBwcSetupPluginFuncTest extends AbstractGradleFuncTest { | ||
|
|
||
| @Rule | ||
| TemporaryFolder remoteRepoDirs = new TemporaryFolder() | ||
|
|
||
| File remoteGitRepo | ||
|
|
||
| def setup() { | ||
| remoteGitRepo = new File(setupGitRemote(), '.git') | ||
|
|
||
| "git clone ${remoteGitRepo.absolutePath}".execute(Collections.emptyList(), testProjectDir.root).waitFor() | ||
| File buildScript = new File(testProjectDir.root, 'remote/build.gradle') | ||
| internalBuild(buildScript) | ||
| buildScript << """ | ||
| apply plugin: 'elasticsearch.internal-distribution-bwc-setup' | ||
| """ | ||
| } | ||
|
|
||
| def "builds distribution from branches via archives assemble"() { | ||
| when: | ||
| def result = gradleRunner(new File(testProjectDir.root, "remote"), | ||
| ":distribution:bwc:bugfix:buildBwcDarwinTar", | ||
| ":distribution:bwc:bugfix:buildBwcOssDarwinTar", | ||
| "-DtestRemoteRepo=" + remoteGitRepo, | ||
| "-Dbwc.remote=origin") | ||
| .build() | ||
| then: | ||
| result.task(":distribution:bwc:bugfix:buildBwcDarwinTar").outcome == TaskOutcome.SUCCESS | ||
| result.task(":distribution:bwc:bugfix:buildBwcOssDarwinTar").outcome == TaskOutcome.SUCCESS | ||
|
|
||
| and: "assemble task triggered" | ||
| result.output.contains("[8.0.1] > Task :distribution:archives:darwin-tar:assemble") | ||
| result.output.contains("[8.0.1] > Task :distribution:archives:oss-darwin-tar:assemble") | ||
| } | ||
|
|
||
| def "bwc distribution archives can be resolved as bwc project artifact"() { | ||
| setup: | ||
| new File(testProjectDir.root, 'remote/build.gradle') << """ | ||
| configurations { | ||
| dists | ||
| } | ||
| dependencies { | ||
| dists project(path: ":distribution:bwc:bugfix", configuration:"darwin-tar") | ||
| } | ||
| tasks.register("resolveDistributionArchive") { | ||
| inputs.files(configurations.dists) | ||
| doLast { | ||
| configurations.dists.files.each { | ||
| println "distfile " + (it.absolutePath - project.rootDir.absolutePath) | ||
| } | ||
| } | ||
| } | ||
| """ | ||
| when: | ||
| def result = gradleRunner(new File(testProjectDir.root, "remote"), | ||
| ":resolveDistributionArchive", | ||
| "-DtestRemoteRepo=" + remoteGitRepo, | ||
| "-Dbwc.remote=origin") | ||
| .build() | ||
| then: | ||
| result.task(":resolveDistributionArchive").outcome == TaskOutcome.SUCCESS | ||
| result.task(":distribution:bwc:bugfix:buildBwcDarwinTar").outcome == TaskOutcome.SUCCESS | ||
|
|
||
| and: "assemble task triggered" | ||
| result.output.contains("[8.0.1] > Task :distribution:archives:darwin-tar:assemble") | ||
| normalizedOutput(result.output) | ||
| .contains("distfile /distribution/bwc/bugfix/build/bwc/checkout-8.0/distribution/archives/darwin-tar/" + | ||
| "build/distributions/elasticsearch-8.0.1-SNAPSHOT-darwin-x86_64.tar.gz") | ||
| } | ||
|
|
||
| def "bwc expanded distribution folder can be resolved as bwc project artifact"() { | ||
| setup: | ||
| new File(testProjectDir.root, 'remote/build.gradle') << """ | ||
| configurations { | ||
| expandedDist | ||
| } | ||
| dependencies { | ||
| expandedDist project(path: ":distribution:bwc:bugfix", configuration:"expanded-darwin-tar") | ||
| } | ||
| tasks.register("resolveExpandedDistribution") { | ||
| inputs.files(configurations.expandedDist) | ||
| doLast { | ||
| configurations.expandedDist.files.each { | ||
| println "distfile " + (it.absolutePath - project.rootDir.absolutePath) | ||
| } | ||
| } | ||
| } | ||
| """ | ||
| when: | ||
| def result = gradleRunner(new File(testProjectDir.root, "remote"), | ||
| ":resolveExpandedDistribution", | ||
| "-DtestRemoteRepo=" + remoteGitRepo, | ||
| "-Dbwc.remote=origin") | ||
| .build() | ||
| then: | ||
| result.task(":resolveExpandedDistribution").outcome == TaskOutcome.SUCCESS | ||
| result.task(":distribution:bwc:bugfix:buildBwcDarwinTar").outcome == TaskOutcome.SUCCESS | ||
|
|
||
| and: "assemble task triggered" | ||
| result.output.contains("[8.0.1] > Task :distribution:archives:darwin-tar:assemble") | ||
| normalizedOutput(result.output) | ||
| .contains("distfile /distribution/bwc/bugfix/build/bwc/checkout-8.0/" + | ||
| "distribution/archives/darwin-tar/build/install") | ||
| } | ||
|
|
||
| File setupGitRemote() { | ||
| URL fakeRemote = getClass().getResource("fake_git/remote") | ||
| File workingRemoteGit = new File(remoteRepoDirs.root, 'remote') | ||
| FileUtils.copyDirectory(new File(fakeRemote.file), workingRemoteGit) | ||
| fakeRemote.file + "/.git" | ||
| gradleRunner(workingRemoteGit, "wrapper").build() | ||
| "git init".execute(Collections.emptyList(), workingRemoteGit).waitFor() | ||
| "git add .".execute(Collections.emptyList(), workingRemoteGit).waitFor() | ||
| 'git commit -m"Initial"'.execute(Collections.emptyList(), workingRemoteGit).waitFor() | ||
| "git checkout -b origin/8.0".execute(Collections.emptyList(), workingRemoteGit).waitFor() | ||
| return workingRemoteGit; | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
21 changes: 21 additions & 0 deletions
21
.../resources/org/elasticsearch/gradle/internal/fake_git/remote/.ci/java-versions.properties
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # | ||
| # Licensed to Elasticsearch under one or more contributor | ||
| # license agreements. See the NOTICE file distributed with | ||
| # this work for additional information regarding copyright | ||
| # ownership. Elasticsearch licenses this file to you under | ||
| # the Apache License, Version 2.0 (the "License"); you may | ||
| # not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| ES_BUILD_JAVA=openjdk14 | ||
| ES_RUNTIME_JAVA=openjdk14 | ||
| GRADLE_TASK=build |
Empty file.
Oops, something went wrong.