Skip to content

Commit

Permalink
SPARK-4579 [WEBUI] Scheduling Delay appears negative
Browse files Browse the repository at this point in the history
Ensure scheduler delay handles unfinished task case, and ensure delay is never negative even due to rounding

Author: Sean Owen <sowen@cloudera.com>

Closes apache#4796 from srowen/SPARK-4579 and squashes the following commits:

ad6713c [Sean Owen] Ensure scheduler delay handles unfinished task case, and ensure delay is never negative even due to rounding
  • Loading branch information
srowen authored and Andrew Or committed Feb 27, 2015
1 parent e60ad2f commit fbc4694
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
Original file line number Diff line number Diff line change
Expand Up @@ -626,15 +626,16 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") {
}

private def getSchedulerDelay(info: TaskInfo, metrics: TaskMetrics): Long = {
val totalExecutionTime = {
if (info.gettingResultTime > 0) {
(info.gettingResultTime - info.launchTime)
val totalExecutionTime =
if (info.gettingResult) {
info.gettingResultTime - info.launchTime
} else if (info.finished) {
info.finishTime - info.launchTime
} else {
(info.finishTime - info.launchTime)
0
}
}
val executorOverhead = (metrics.executorDeserializeTime +
metrics.resultSerializationTime)
totalExecutionTime - metrics.executorRunTime - executorOverhead
math.max(0, totalExecutionTime - metrics.executorRunTime - executorOverhead)
}
}

0 comments on commit fbc4694

Please sign in to comment.