-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-41091][BUILD][3.2] Fix Docker release tool for branch-3.2 #38643
Conversation
cc @HyukjinKwon @srowen @zero323 who was involved in #34728. I'm not familiar with R at all so please check if this is actually doing the correct thing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SPARK-37474 is Apache Spark 3.3.0. That cannot be the root cause of any issues at branch-3.2. Given that, I'm wondering if you want to backport some branch-3.3 patch to branch-3.2 because of the latest OS change?
Or, are you trying to release branch-3.2
by using Apache Spark master branch release script, @sunchao ?
I'm not sure what patch to backport: I only checked what makes the
No, I'm releasing from |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it. +1, LGTM.
### What changes were proposed in this pull request? This tries to fix `do-release-docker.sh` for branch-3.2. ### Why are the changes needed? Currently the following error will occur if running the script in `branch-3.2`: ``` #5 917.4 g++ -std=gnu++14 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o testthat.so init.o reassign.o test-catch.o test-example.o test-runner.o -L/usr/lib/R/lib -lR #5 917.5 installing to /usr/local/lib/R/site-library/00LOCK-testthat/00new/testthat/libs #5 917.5 ** R #5 917.5 ** inst #5 917.5 ** byte-compile and prepare package for lazy loading #5 924.4 ** help #5 924.6 *** installing help indices #5 924.7 *** copying figures #5 924.7 ** building package indices #5 924.9 ** installing vignettes #5 924.9 ** testing if installed package can be loaded from temporary location #5 925.1 ** checking absolute paths in shared objects and dynamic libraries #5 925.1 ** testing if installed package can be loaded from final location #5 925.5 ** testing if installed package keeps a record of temporary installation path #5 925.5 * DONE (testthat) #5 925.8 ERROR: dependency 'pkgdown' is not available for package 'devtools' #5 925.8 * removing '/usr/local/lib/R/site-library/devtools' #5 925.8 #5 925.8 The downloaded source packages are in #5 925.8 '/tmp/Rtmp3nJI60/downloaded_packages' #5 925.8 Warning messages: #5 925.8 1: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : #5 925.8 installation of package 'textshaping' had non-zero exit status #5 925.8 2: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : #5 925.8 installation of package 'ragg' had non-zero exit status #5 925.8 3: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : #5 925.8 installation of package 'pkgdown' had non-zero exit status #5 925.8 4: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : #5 925.8 installation of package 'devtools' had non-zero exit status #5 926.0 Error in loadNamespace(x) : there is no package called 'devtools' #5 926.0 Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart #5 926.0 Execution halted ``` The same error doesn't happen on master. I checked the diff between the two and it seems the following line: ``` $APT_INSTALL libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev && \ ``` introduced in #34728 made the difference. I verified that after adding the line, `do-release-docker.sh` (dry run mode) was able to finish successfully. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually Closes #38643 from sunchao/fix-docker-release. Authored-by: Chao Sun <sunchao@apple.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Since this is irrelevant to GitHub Action CI, I merged this to branch-3.2. |
LGTM. Thanks @sunchao! |
Thanks @dongjoon-hyun @zero323 ! I think I'm unblocked from 3.2.3 release now. |
It's great. I saw the tag. Thank you! |
### What changes were proposed in this pull request? This tries to fix `do-release-docker.sh` for branch-3.2. ### Why are the changes needed? Currently the following error will occur if running the script in `branch-3.2`: ``` apache#5 917.4 g++ -std=gnu++14 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o testthat.so init.o reassign.o test-catch.o test-example.o test-runner.o -L/usr/lib/R/lib -lR apache#5 917.5 installing to /usr/local/lib/R/site-library/00LOCK-testthat/00new/testthat/libs apache#5 917.5 ** R apache#5 917.5 ** inst apache#5 917.5 ** byte-compile and prepare package for lazy loading apache#5 924.4 ** help apache#5 924.6 *** installing help indices apache#5 924.7 *** copying figures apache#5 924.7 ** building package indices apache#5 924.9 ** installing vignettes apache#5 924.9 ** testing if installed package can be loaded from temporary location apache#5 925.1 ** checking absolute paths in shared objects and dynamic libraries apache#5 925.1 ** testing if installed package can be loaded from final location apache#5 925.5 ** testing if installed package keeps a record of temporary installation path apache#5 925.5 * DONE (testthat) apache#5 925.8 ERROR: dependency 'pkgdown' is not available for package 'devtools' apache#5 925.8 * removing '/usr/local/lib/R/site-library/devtools' apache#5 925.8 apache#5 925.8 The downloaded source packages are in apache#5 925.8 '/tmp/Rtmp3nJI60/downloaded_packages' apache#5 925.8 Warning messages: apache#5 925.8 1: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : apache#5 925.8 installation of package 'textshaping' had non-zero exit status apache#5 925.8 2: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : apache#5 925.8 installation of package 'ragg' had non-zero exit status apache#5 925.8 3: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : apache#5 925.8 installation of package 'pkgdown' had non-zero exit status apache#5 925.8 4: In install.packages(c("curl", "xml2", "httr", "devtools", "testthat", : apache#5 925.8 installation of package 'devtools' had non-zero exit status apache#5 926.0 Error in loadNamespace(x) : there is no package called 'devtools' apache#5 926.0 Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart apache#5 926.0 Execution halted ``` The same error doesn't happen on master. I checked the diff between the two and it seems the following line: ``` $APT_INSTALL libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev && \ ``` introduced in apache#34728 made the difference. I verified that after adding the line, `do-release-docker.sh` (dry run mode) was able to finish successfully. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually Closes apache#38643 from sunchao/fix-docker-release. Authored-by: Chao Sun <sunchao@apple.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
What changes were proposed in this pull request?
This tries to fix
do-release-docker.sh
for branch-3.2.Why are the changes needed?
Currently the following error will occur if running the script in
branch-3.2
:The same error doesn't happen on master. I checked the diff between the two and it seems the following line:
introduced in #34728 made the difference.
I verified that after adding the line,
do-release-docker.sh
(dry run mode) was able to finish successfully.Does this PR introduce any user-facing change?
No.
How was this patch tested?
Manually