Skip to content

Commit e6f276e

Browse files
committed
[SPARK-32491][INFRA] Do not install SparkR in test-only mode in testing script
This PR proposes to skip SparkR installation that is to run R linters (see SPARK-8505) in the test-only mode at `dev/run-tests.py` script. As of SPARK-32292, the test-only mode in `dev/run-tests.py` was introduced, for example: ``` dev/run-tests.py --modules sql,core ``` which only runs the relevant tests and does not run other tests such as linters. Therefore, we don't need to install SparkR when `--modules` are specified. GitHub Actions build is currently failed as below: ``` ERROR: this R is version 3.4.4, package 'SparkR' requires R >= 3.5 [error] running /home/runner/work/spark/spark/R/install-dev.sh ; received return code 1 ``` For some reasons, looks GitHub Actions started to have R 3.4.4 installed by default; however, R 3.4 was dropped as of SPARK-32073. When SparkR tests are not needed, GitHub Actions still builds SparkR with a low R version and it causes the test failure. This PR partially fixes it by avoid the installation of SparkR. No, dev-only. GitHub Actions tests should run to confirm this fix is correct. Closes apache#29300 from HyukjinKwon/install-r. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
1 parent 1aa2cb7 commit e6f276e

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

dev/run-tests.py

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -556,13 +556,14 @@ def main():
556556
" install one and retry.")
557557
sys.exit(2)
558558

559-
java_version = determine_java_version(java_exe)
560-
561-
# install SparkR
562-
if which("R"):
563-
run_cmd([os.path.join(SPARK_HOME, "R", "install-dev.sh")])
564-
else:
565-
print("Cannot install SparkR as R was not found in PATH")
559+
# Install SparkR
560+
should_only_test_modules = opts.modules is not None
561+
if not should_only_test_modules:
562+
# If tests modules are specified, we will not run R linter.
563+
if which("R"):
564+
run_cmd([os.path.join(SPARK_HOME, "R", "install-dev.sh")])
565+
else:
566+
print("Cannot install SparkR as R was not found in PATH")
566567

567568
if os.environ.get("AMPLAB_JENKINS"):
568569
# if we're on the Amplab Jenkins build servers setup variables
@@ -587,7 +588,6 @@ def main():
587588
changed_modules = None
588589
test_modules = None
589590
changed_files = None
590-
should_only_test_modules = opts.modules is not None
591591
included_tags = []
592592
excluded_tags = []
593593
if should_only_test_modules:

0 commit comments

Comments
 (0)