Skip to content

Commit 1ec67d6

Browse files
pawel-gudel-eliatrapeternied
pawel-gudel-eliatra
and
peternied
authored
[Backport] Parallel test jobs for CI (#2861) (#2936)
* [Enhancement] Parallel test jobs for CI (#2861) * Split multiple tests into separate gradle tasks. * Tasks are configured in "splitTestConfig" map in build.gradle file. Map allows to use all patterns from TestFilter like: includeTestsMatching, excludeTestsMatching, includeTest etc. * Tasks are automatically generated from "splitTestConfig" map. * Two new Gradle tasks: listTasksAsJSON and listTasksAsParam to output task names to console. First one outputs them as a JSON and second - in gradlew "-x <TASK>" format to use in CLI. * Patterns included in tasks are automatically excluded from main "test" task but at the same time generated tasks are dependencies for "test". Running "gradlew test" will run whole suite at once. * CI pipeline has been configured to accomodate all changes. * New 'master' task to generate list of jobs to run in parallel. * Updated matrix strategy to include task name to start. Signed-off-by: Pawel Gudel <pawel.gudel@eliatra.com> (cherry picked from commit e4f4817) Signed-off-by: Pawel Gudel <pawel.gudel@eliatra.com> * Generalize Backwards Compatibility tests so we can test from any version to any version With an issue reported indicating that there are serialization issue between 1.3 and 2.X, making sure that we can reproduce the errors. This new workflow(s) will make sure that we aren't breaking BWC with changes we are adding to 2.X and will give us the flexibility to test certain migration workflows. Fixing an issue where the BWC tests were not actually building or executing causing the clusters to spin up and then immediately spin down. We will need to invest more energy into running multiple kinds of security plugin specific scenarios through the test system. Signed-off-by: Peter Nied <petern@amazon.com> * Remove seperate integration test workflow Signed-off-by: Peter Nied <petern@amazon.com> * Adjust BWC setting to point to correct BWC versions Signed-off-by: Peter Nied <petern@amazon.com> * Fix spotless issues Signed-off-by: Peter Nied <petern@amazon.com> --------- Signed-off-by: Pawel Gudel <pawel.gudel@eliatra.com> Signed-off-by: Peter Nied <petern@amazon.com> Co-authored-by: Peter Nied <petern@amazon.com>
1 parent e1fc42f commit 1ec67d6

File tree

7 files changed

+378
-117
lines changed

7 files changed

+378
-117
lines changed

‎.github/actions/create-bwc-build/action.yaml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: 'Create a backwards compatible ready build'
2+
description: 'Checkouts the official version of a the Security plugin and builds it so it can be used for BWC tests'
3+
4+
inputs:
5+
plugin-branch:
6+
description: 'The branch of the plugin that should be built, e.g "2.2", "1.x"'
7+
required: true
8+
9+
outputs:
10+
built-version:
11+
description: 'The version of OpenSearch that was associated with this branch'
12+
value: ${{ steps.get-opensearch-version.outputs.version }}
13+
14+
runs:
15+
using: "composite"
16+
steps:
17+
- name: Enable Longpaths if on Windows
18+
if: ${{ runner.os == 'Windows' }}
19+
run: git config --system core.longpaths true
20+
shell: pwsh
21+
22+
- uses: actions/checkout@v3
23+
with:
24+
repository: opensearch-project/security
25+
ref: ${{ inputs.plugin-branch }}
26+
path: ${{ inputs.plugin-branch }}
27+
28+
- name: Build
29+
uses: gradle/gradle-build-action@v2
30+
with:
31+
arguments: assemble -Dbuild.snapshot=false
32+
build-root-directory: ${{ inputs.plugin-branch }}
33+
34+
- id: get-opensearch-version
35+
uses: peternied/get-opensearch-version@v1
36+
with:
37+
working-directory: ${{ inputs.plugin-branch }}
38+
39+
- name: Copy current distro into the expected folder
40+
run: |
41+
mkdir -p ./bwc-test/src/test/resources/${{ steps.get-opensearch-version.outputs.version }}
42+
cp ${{ inputs.plugin-branch }}/build/distributions/opensearch-security-${{ steps.get-opensearch-version.outputs.version }}.zip ./bwc-test/src/test/resources/${{ steps.get-opensearch-version.outputs.version }}
43+
shell: bash

‎.github/actions/run-bwc-suite/action.yaml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
name: 'Runs the backward bompatiblity test suite'
2+
description: 'Tests backwards compability between a previous and next version of this plugin'
3+
4+
inputs:
5+
plugin-previous-branch:
6+
description: 'The branch of the plugin that should be built for the previous version, e.g "2.2", "1.x"'
7+
required: true
8+
9+
plugin-next-branch:
10+
description: 'The branch of the plugin that should be built for the next version, e.g "2.3", "main"'
11+
required: true
12+
13+
report-artifact-name:
14+
description: 'The name of the artifacts for this run, e.g. "BWC-2.1-to-2.4-results"'
15+
required: true
16+
17+
runs:
18+
using: "composite"
19+
steps:
20+
21+
- id: build-previous
22+
uses: ./.github/actions/create-bwc-build
23+
with:
24+
plugin-branch: ${{ inputs.plugin-previous-branch }}
25+
26+
- id: build-next
27+
uses: ./.github/actions/create-bwc-build
28+
with:
29+
plugin-branch: ${{ inputs.plugin-next-branch }}
30+
31+
- name: Run BWC tests
32+
uses: gradle/gradle-build-action@v2
33+
with:
34+
arguments: |
35+
bwcTestSuite
36+
-Dtests.security.manager=false
37+
-Dbwc.version.previous=${{ steps.build-previous.outputs.built-version }}
38+
-Dbwc.version.next=${{ steps.build-next.outputs.built-version }} -i
39+
build-root-directory: bwc-test
40+
41+
- uses: actions/upload-artifact@v3
42+
if: always()
43+
with:
44+
name: ${{ inputs.report-artifact-name }}
45+
path: |
46+
./bwc-test/build/reports/

‎.github/workflows/bwc-tests.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: Backwards Compability Suite
2+
3+
on: [workflow_dispatch]
4+
5+
jobs:
6+
last-supported-major-to-current:
7+
name: Make sure that the last supported major version can move to the most current version
8+
runs-on: ubuntu-latest
9+
10+
steps:
11+
- uses: actions/setup-java@v1
12+
with:
13+
java-version: 11
14+
15+
- name: Checkout Security Repo
16+
uses: actions/checkout@v2
17+
18+
- id: build-previous
19+
uses: ./.github/actions/run-bwc-suite
20+
with:
21+
plugin-previous-branch: "1.3"
22+
plugin-next-branch: "2.x"
23+
report-artifact-name: BWC-Last-Supported-Major
24+
25+
current-to-next-unreleased-major:
26+
name: Make sure that the current version is compatible with the next major version
27+
runs-on: ubuntu-latest
28+
29+
steps:
30+
- uses: actions/setup-java@v1
31+
with:
32+
java-version: 11
33+
34+
- name: Checkout Security Repo
35+
uses: actions/checkout@v2
36+
37+
- id: build-previous
38+
uses: ./.github/actions/run-bwc-suite
39+
with:
40+
plugin-previous-branch: "2.x"
41+
plugin-next-branch: "main"
42+
report-artifact-name: BWC-Next-Major

‎.github/workflows/ci.yml‎

Lines changed: 44 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,34 @@ env:
66
GRADLE_OPTS: -Dhttp.keepAlive=false
77

88
jobs:
9-
build:
10-
name: build
9+
generate-test-list:
10+
runs-on: ubuntu-latest
11+
outputs:
12+
separateTestsNames: ${{ steps.set-matrix.outputs.separateTestsNames }}
13+
steps:
14+
- name: Set up JDK for build and test
15+
uses: actions/setup-java@v2
16+
with:
17+
distribution: temurin # Temurin is a distribution of adoptium
18+
java-version: 17
19+
20+
- name: Checkout security
21+
uses: actions/checkout@v2
22+
23+
- name: Generate list of tasks
24+
id: set-matrix
25+
run: |
26+
echo "separateTestsNames=$(./gradlew listTasksAsJSON -q --console=plain | tail -n 1)" >> $GITHUB_OUTPUT
27+
28+
test:
29+
name: test
30+
needs: generate-test-list
1131
strategy:
1232
fail-fast: false
1333
matrix:
34+
gradle_task: ${{ fromJson(needs.generate-test-list.outputs.separateTestsNames) }}
35+
platform: [windows-latest, ubuntu-latest]
1436
jdk: [11, 17]
15-
platform: ["ubuntu-latest", "windows-latest", "macos-latest"]
1637
runs-on: ${{ matrix.platform }}
1738

1839
steps:
@@ -29,11 +50,8 @@ jobs:
2950
uses: gradle/gradle-build-action@v2
3051
with:
3152
arguments: |
32-
build test -Dbuild.snapshot=false
33-
-x spotlessCheck
34-
-x checkstyleMain
35-
-x checkstyleTest
36-
-x spotbugsMain
53+
${{ matrix.gradle_task }} -Dbuild.snapshot=false
54+
-x test
3755
3856
- name: Coverage
3957
uses: codecov/codecov-action@v1
@@ -53,25 +71,27 @@ jobs:
5371
run: echo "Check the artifact ${{ matrix.platform }}-JDK${{ matrix.jdk }}-reports for detailed test results"
5472

5573
backward-compatibility:
56-
runs-on: ubuntu-latest
74+
strategy:
75+
fail-fast: false
76+
matrix:
77+
jdk: [11, 17]
78+
platform: [ubuntu-latest, windows-latest]
79+
runs-on: ${{ matrix.platform }}
80+
5781
steps:
58-
- uses: actions/checkout@v2
5982
- uses: actions/setup-java@v1
6083
with:
61-
java-version: 11
62-
- run: ./gradlew clean build -Dbuild.snapshot=false -x test
63-
- run: |
64-
echo "Running backwards compatibility tests ..."
65-
security_plugin_version_no_snapshot=$(./gradlew properties -q | grep -E '^version:' | awk '{print $2}' | sed 's/-SNAPSHOT//g')
66-
bwc_version=2.8.0
67-
cp -r build/ ./bwc-test/
68-
mkdir ./bwc-test/src/test/resources/security_plugin_version_no_snapshot
69-
cp build/distributions/opensearch-security-${security_plugin_version_no_snapshot}.zip ./bwc-test/src/test/resources/${security_plugin_version_no_snapshot}
70-
mkdir bwc-test/src/test/resources/${bwc_version}.0
71-
wget https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${bwc_version}/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-security-${bwc_version}.0.zip
72-
mv opensearch-security-${bwc_version}.0.zip bwc-test/src/test/resources/${bwc_version}.0/
73-
cd bwc-test/
74-
./gradlew bwcTestSuite -Dtests.security.manager=false
84+
java-version: ${{ matrix.jdk }}
85+
86+
- name: Checkout Security Repo
87+
uses: actions/checkout@v2
88+
89+
- id: build-previous
90+
uses: ./.github/actions/run-bwc-suite
91+
with:
92+
plugin-previous-branch: "2.8"
93+
plugin-next-branch: "2.x"
94+
report-artifact-name: bwc-${{ matrix.platform }}-jdk${{ matrix.jdk }}
7595

7696
code-ql:
7797
runs-on: ubuntu-latest

0 commit comments

Comments
 (0)