[SPARK-29596][Web UI] Task duration not updating for running tasks #27026
Conversation
| @@ -848,7 +848,7 @@ private[ui] class TaskPagedTable( | |||
| </div> | |||
| </td> | |||
| <td>{UIUtils.formatDate(task.launchTime)}</td> | |||
| <td>{formatDuration(task.taskMetrics.map(_.executorRunTime))}</td> | |||
There was a problem hiding this comment.
Hi @MrDLontheway , please note that this change is intentionally introduced in #23081.
Thank you for your reminder, during the task running process, because the executive run time is not updated, the duration will not be updated. I will seek other solutions |
I know that executor run time doesn't update according to your description, but I also want to know the root cause why it doesn't change. |
|
@MrDLontheway could you provide more details and screenshot for the issue? |
|
|
There was a problem hiding this comment.
I'm not sure about this change. There are a lot of more information that is used from metrics to create TaskDataWrapper, so, the other metrics are not updated? If this is the case the problem could be we are not updating metrics in the right way and we have to change it and not this specific metric only.
| var duration: Long = 0 | ||
| var executorRunTime: Long = 0 | ||
|
|
||
| if (info.finished) { |
There was a problem hiding this comment.
vars are not very cool in scala... how about something like this?
val (duration, executorRunTime) = if (info.finished) {
(info.duration, metrics.executorRunTime)
} else {
val timeRunning = info.timeRunning(lastUpdateTime.getOrElse(System.currentTimeMillis()))
(timeRunning, timeRunning)
}
|
Jenkins ok to test |
|
Test build #121933 has finished for PR 27026 at commit
|
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
What changes were proposed in this pull request?
Changed the source of web ui duration information
Why are the changes needed?
Task duration not updating for running tasks
Does this PR introduce any user-facing change?
no
How was this patch tested?
cat web ui