Commit 5ffc73e
[SPARK-5074] [CORE] [TESTS] Fix the flakey test 'run shuffle with map stage failure' in DAGSchedulerSuite
Test failure: https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.2,label=centos/2240/testReport/junit/org.apache.spark.scheduler/DAGSchedulerSuite/run_shuffle_with_map_stage_failure/
This is because many tests share the same `JobListener`. Because after each test, `scheduler` isn't stopped. So actually it's still running. When running the test `run shuffle with map stage failure`, some previous test may trigger [ResubmitFailedStages](https://github.com/apache/spark/blob/ebc25a4ddfe07a67668217cec59893bc3b8cf730/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala#L1120) logic, and report `jobFailed` and override the global `failure` variable.
This PR uses `after` to call `scheduler.stop()` for each test.
Author: zsxwing <zsxwing@gmail.com>
Closes #5903 from zsxwing/SPARK-5074 and squashes the following commits:
1e6f13e [zsxwing] Fix the flakey test 'run shuffle with map stage failure' in DAGSchedulerSuite1 parent b83091a commit 5ffc73e
File tree
1 file changed
+6
-1
lines changed- core/src/test/scala/org/apache/spark/scheduler
1 file changed
+6
-1
lines changedLines changed: 6 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
177 | 181 | | |
178 | 182 | | |
179 | 183 | | |
| |||
261 | 265 | | |
262 | 266 | | |
263 | 267 | | |
264 | | - | |
| 268 | + | |
265 | 269 | | |
| 270 | + | |
266 | 271 | | |
267 | 272 | | |
268 | 273 | | |
| |||
0 commit comments