Skip to content

refactor: use download_pingcap_oci_artifact.sh instead of download_pingcap_artifact.sh #4136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wuhuizuo wants to merge 3 commits into
base: main
Choose a base branch
from
Open

refactor: use download_pingcap_oci_artifact.sh instead of download_pingcap_artifact.sh #4136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d2ec82c
Add utils container to all integration test pod templates
wuhuizuo Feb 1, 2026
97f31f4
refactor: use download_pingcap_oci_artifact.sh in pipeline groovy files
wuhuizuo Feb 1, 2026
9fa0a81
fix: auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions pipelines/pingcap-qe/tidb-test/release-6.0/ghpr_integration_mysql_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
}
options {
timeout(time: 70, unit: 'MINUTES')
Expand Down Expand Up @@ -66,10 +66,12 @@ pipeline {
dir('tidb') {
cache(path: "./bin", includes: '**/*', key: "ws/${BUILD_TAG}/dependencies") {
sh label: 'tidb-server', script: 'ls bin/tidb-server || make'
}
container("utils") {
sh label: 'download binary', script: """
chmod +x ${WORKSPACE}/scripts/artifacts/*.sh
${WORKSPACE}/scripts/artifacts/download_pingcap_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
mv third_bin/* bin/
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}

mv pd-server bin/
Comment on lines +72 to +74
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The mv command has incorrect indentation and is incomplete. It should be properly indented to be part of the shell script. Additionally, the download command fetches both pd and tikv artifacts, but only pd-server is being moved. The tikv-server also needs to be moved to the bin directory for subsequent steps to succeed.

                            ${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
                            mv pd-server tikv-server bin/

ls -alh bin/
"""
}
Expand Down Expand Up @@ -104,14 +106,14 @@ pipeline {
sh label: "print version", script: """
pwd && ls -alh
ls bin/tidb-server && chmod +x bin/tidb-server && ./bin/tidb-server -V
ls bin/pd-server && chmod +x bin/pd-server && ./bin/pd-server -V
ls bin/tikv-server && chmod +x bin/tikv-server && ./bin/tikv-server -V
ls bin/pd-server && chmod +x bin/pd-server && ./pd-server -V
ls bin/tikv-server && chmod +x bin/tikv-server && ./tikv-server -V
Comment on lines +109 to +110
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The paths to the pd-server and tikv-server binaries are incorrect. The current working directory for this step is tidb, and the binaries are located in the bin subdirectory. Therefore, the commands should be ./bin/pd-server -V and ./bin/tikv-server -V respectively.

                                        ls bin/pd-server && chmod +x bin/pd-server && ./bin/pd-server -V
                                        ls bin/tikv-server && chmod +x bin/tikv-server && ./bin/tikv-server -V

"""
}
}
dir('tidb-test/mysql_test') {
sh """
mkdir -p bin

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The mkdir -p bin command has been removed. The subsequent mv command will fail if the bin directory doesn't exist. It's safer to restore this command to ensure the directory is created before moving files into it.

                                    mkdir -p bin

mv ${WORKSPACE}/tidb/bin/* bin/ && chmod +x bin/*
ls -alh bin/
"""
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap-qe/tidb-test/release-6.0/pod-ghpr_integration_mysql_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,16 @@ spec:
limits:
memory: 256Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
16 changes: 9 additions & 7 deletions pipelines/pingcap-qe/tidb-test/release-6.2/ghpr_integration_mysql_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
}
options {
timeout(time: 45, unit: 'MINUTES')
Expand Down Expand Up @@ -66,10 +66,12 @@ pipeline {
dir('tidb') {
cache(path: "./bin", includes: '**/*', key: "ws/${BUILD_TAG}/dependencies") {
sh label: 'tidb-server', script: 'ls bin/tidb-server || make'
}
container("utils") {
sh label: 'download binary', script: """
chmod +x ${WORKSPACE}/scripts/artifacts/*.sh
${WORKSPACE}/scripts/artifacts/download_pingcap_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
mv third_bin/* bin/
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}

mv pd-server bin/
ls -alh bin/
"""
}
Expand Down Expand Up @@ -108,14 +110,14 @@ pipeline {
sh label: "print version", script: """
pwd && ls -alh
ls bin/tidb-server && chmod +x bin/tidb-server && ./bin/tidb-server -V
ls bin/pd-server && chmod +x bin/pd-server && ./bin/pd-server -V
ls bin/tikv-server && chmod +x bin/tikv-server && ./bin/tikv-server -V
ls bin/pd-server && chmod +x bin/pd-server && ./pd-server -V
ls bin/tikv-server && chmod +x bin/tikv-server && ./tikv-server -V
"""
}
}
dir('tidb-test/mysql_test') {
sh """
mkdir -p bin

mv ${WORKSPACE}/tidb/bin/* bin/ && chmod +x bin/*
ls -alh bin/
"""
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap-qe/tidb-test/release-6.2/pod-ghpr_integration_mysql_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,16 @@ spec:
limits:
memory: 256Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pod-pull_e2e_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pod-pull_integration_common_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pod-pull_integration_copr_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pod-pull_integration_ddl_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pod-pull_integration_jdbc_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
10 changes: 10 additions & 0 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pod-pull_integration_mysql_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ spec:
limits:
memory: 128Mi
cpu: 100m
- name: utils
image: ghcr.io/pingcap-qe/cd/utils/release:v2025.10.12-7-gfdd779c
tty: true
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
cpu: "1"
memory: 4Gi
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
Expand Down
33 changes: 18 additions & 15 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pull_e2e_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
}
options {
timeout(time: 40, unit: 'MINUTES')
Expand Down Expand Up @@ -53,18 +53,21 @@ pipeline {
stage('Prepare') {
steps {
dir('tidb') {
sh label: 'tidb-server', script: '[ -f bin/tidb-server ] || make'
retry(3) {
sh label: 'download binary', script: """
chmod +x \${WORKSPACE}/scripts/artifacts/*.sh
\${WORKSPACE}/scripts/artifacts/download_pingcap_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
mv third_bin/tikv-server bin/
mv third_bin/pd-server bin/
ls -alh bin/
chmod +x bin/*
./bin/tikv-server -V
./bin/pd-server -V
"""
container("golang") {
sh label: 'tidb-server', script: '[ -f bin/tidb-server ] || make'
}
dir("bin") {
container("utils") {
retry(3) {
sh label: 'download binary', script: """
${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
ls -alh
chmod +x *
./tikv-server -V
./pd-server -V
"""
}
}
}
}
}
Expand All @@ -76,8 +79,8 @@ pipeline {
sh label: 'check version', script: """
ls -alh bin/
./bin/tidb-server -V
./bin/tikv-server -V
./bin/pd-server -V
./tikv-server -V
./pd-server -V
"""
sh label: 'test graceshutdown', script: """
cd tests/graceshutdown && make
Expand Down
25 changes: 13 additions & 12 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pull_integration_common_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
GITHUB_TOKEN = credentials('github-bot-token')
}
options {
Expand Down Expand Up @@ -69,13 +69,14 @@ pipeline {
container("golang") {
sh label: 'tidb-server', script: '[ -f bin/tidb-server ] || make'
retry(3) {
sh label: 'download thirdparty binary', script: """
chmod +x ${WORKSPACE}/scripts/artifacts/*.sh
${WORKSPACE}/scripts/artifacts/download_pingcap_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
mv third_bin/tikv-server bin/
mv third_bin/pd-server bin/
./bin/tikv-server -V
./bin/pd-server -V
dir("bin") { container("utils") { sh label: 'download binary', script: """
container("utils") {
sh label: 'download binary', script: "${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}"



./tikv-server -V
./pd-server -V
"""
}
Comment on lines +72 to 81
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This script block is syntactically incorrect. It appears to have a nested Groovy container and sh step within the shell script string, which will cause the pipeline to fail. The entire block needs to be refactored to have a valid structure, with only shell commands inside the sh script's triple quotes.

}
Expand All @@ -88,8 +89,8 @@ pipeline {
cd randgen-test && ./build.sh && cd ..
mkdir -p bin
cp -r ../tidb/bin/* bin/ && chmod +x bin/*
./bin/pd-server -V
./bin/tikv-server -V
./pd-server -V
./tikv-server -V
./bin/tidb-server -V
"""
}
Expand Down Expand Up @@ -123,8 +124,8 @@ pipeline {
cache(path: "./", includes: '**/*', key: "ws/${BUILD_TAG}/tidb-test") {
sh """
ls -alh bin/
./bin/pd-server -V
./bin/tikv-server -V
./pd-server -V
./tikv-server -V
./bin/tidb-server -V
"""
container("golang") {
Expand Down
19 changes: 10 additions & 9 deletions pipelines/pingcap/tidb/release-6.5-20241101-v6.5.7/pull_integration_copr_test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ pipeline {
}
}
environment {
FILE_SERVER_URL = 'http://fileserver.pingcap.net'
OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub'
GITHUB_TOKEN = credentials('github-bot-token')
}
options {
Expand Down Expand Up @@ -72,15 +72,16 @@ pipeline {
container("golang") {
retry(2) {
sh label: 'download binary', script: """
chmod +x ${WORKSPACE}/scripts/artifacts/*.sh
${WORKSPACE}/scripts/artifacts/download_pingcap_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}
container("utils") {
sh label: 'download binary', script: "${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh --pd=${REFS.base_ref} --tikv=${REFS.base_ref}"
rm -rf bin/ && mkdir -p bin/
mv third_bin/tikv-server bin/
mv third_bin/pd-server bin/
mkdir -p bin

mv pd-server bin/
ls -alh bin/
chmod +x bin/*
./bin/tikv-server -V
./bin/pd-server -V
./tikv-server -V
./pd-server -V
"""
Comment on lines 74 to 85
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This script block is syntactically incorrect due to a nested container and sh step within the script string. Additionally, the logic is flawed: it attempts to move pd-server but not tikv-server, and then tries to check the version of tikv-server from a location it hasn't been moved to. This will cause multiple failures.

}
}
Expand All @@ -93,8 +94,8 @@ pipeline {
dir('tidb') {
sh label: 'check version', script: """
ls -alh bin/
./bin/tikv-server -V
./bin/pd-server -V
./tikv-server -V
./pd-server -V
"""
}
dir('tikv-copr-test') {
Expand Down
Loading