From e82fcf817ad0f535a034d17f52c7431285ef3a77 Mon Sep 17 00:00:00 2001 From: HyukjinKwon Date: Fri, 24 Jul 2020 08:52:30 -0700 Subject: [PATCH] [SPARK-32408][BUILD] Enable crossPaths back to prevent side effects ### What changes were proposed in this pull request? This PR proposes to enable `corssPaths` back for now to match with the build as it was. It still indeterministically doesn't run JUnit tests given my observation, and this PR basically reverts the partial fix from https://github.com/apache/spark/pull/29057. See also https://github.com/apache/spark/pull/29205 for the full context. ### Why are the changes needed? To prevent the side effects from crossPaths such as SPARK_PREPEND_CLASSES or tests that run conditionally if the test classes are present in PySpark. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? Manually tested: ```bash build/sbt -Phadoop-2.7 -Phive -Phive-2.3 -Phive-thriftserver -DskipTests clean test:package ./python/run-tests --python-executable=python3 --testname="pyspark.sql.tests.test_dataframe QueryExecutionListenerTests" ``` Closes #29218 from HyukjinKwon/SPARK-32408-1. Authored-by: HyukjinKwon Signed-off-by: Dongjoon Hyun --- project/SparkBuild.scala | 2 -- 1 file changed, 2 deletions(-) diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index 197880b335969..7a9634ac0fcbb 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -1029,8 +1029,6 @@ object TestSettings { // Show full stack trace and duration in test cases. testOptions in Test += Tests.Argument("-oDF"), testOptions in Test += Tests.Argument(TestFrameworks.JUnit, "-v", "-a"), - // Required to detect Junit tests for each project, see also https://github.com/sbt/junit-interface/issues/35 - crossPaths := false, // Enable Junit testing. libraryDependencies += "com.novocode" % "junit-interface" % "0.11" % "test", // `parallelExecutionInTest` controls whether test suites belonging to the same SBT project