From 2df1488ac1b00f4a1697eadb6e45b501945167d6 Mon Sep 17 00:00:00 2001 From: HyukjinKwon Date: Fri, 24 Jul 2020 08:52:30 -0700 Subject: [PATCH] [SPARK-32408][BUILD] Enable crossPaths back to prevent side effects ### What changes were proposed in this pull request? This PR proposes to enable `corssPaths` back for now to match with the build as it was. It still indeterministically doesn't run JUnit tests given my observation, and this PR basically reverts the partial fix from https://github.com/apache/spark/pull/29057. See also https://github.com/apache/spark/pull/29205 for the full context. ### Why are the changes needed? To prevent the side effects from crossPaths such as SPARK_PREPEND_CLASSES or tests that run conditionally if the test classes are present in PySpark. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? Manually tested: ```bash build/sbt -Phadoop-2.7 -Phive -Phive-2.3 -Phive-thriftserver -DskipTests clean test:package ./python/run-tests --python-executable=python3 --testname="pyspark.sql.tests.test_dataframe QueryExecutionListenerTests" ``` Closes #29218 from HyukjinKwon/SPARK-32408-1. Authored-by: HyukjinKwon Signed-off-by: Dongjoon Hyun --- project/SparkBuild.scala | 2 -- 1 file changed, 2 deletions(-) diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index c709d9ea4b9bf..12b8ede59c0c0 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -934,8 +934,6 @@ object TestSettings { // Show full stack trace and duration in test cases. testOptions in Test += Tests.Argument("-oDF"), testOptions in Test += Tests.Argument(TestFrameworks.JUnit, "-v", "-a"), - // Required to detect Junit tests for each project, see also https://github.com/sbt/junit-interface/issues/35 - crossPaths := false, // Enable Junit testing. libraryDependencies += "com.novocode" % "junit-interface" % "0.11" % "test", // `parallelExecutionInTest` controls whether test suites belonging to the same SBT project