SPARK-2298: Show stage attempt in UI #1384

tsudukim · 2014-07-12T03:00:31Z

Added attempt ID column into stage page of webUI.
Added attemptId handling code into StageInfo, JsonProtocol.
Modified DAGScheduler to identify stages whose stageId is same but attemptId is different.
Modified testcode for stage attempt ID.

AmplabJenkins · 2014-07-12T03:01:20Z

Can one of the admins verify this patch?

tsudukim · 2014-07-12T03:03:48Z

Attempt Id shows up in web ui. Submitted and Duration became individual value to stage attempts.

pwendell · 2014-07-12T03:18:37Z

To make this a bit more concise, what about having one column on the left whose header is ID: Attempt and separating them with a colon. Current the word "Stage" is redundant there.

pwendell · 2014-07-12T03:25:24Z

/cc @rxin who is interested in this.

tsudukim · 2014-07-12T05:13:35Z

@pwendell Thank you for your response. You mean like this?

andrewor14 · 2014-07-12T05:21:42Z

Hm the latest screenshot looks a little funky to me.

Most stages will only have 1 attempt, so I think it makes sense to only show the attempt if this is not the first one. Something like:

ID
---------------
4
2
3 (Attempt 2)
1
0

tsudukim · 2014-07-12T06:15:56Z

@andrewor14 Thank you for your comment.
I think it is more weird if the display style of ID/attempt changes by conditions.
Surely most stages will only have 1 attempt, but as the set of stage id and stage attempt id is the identifier of the taskset, I'd like to show the attempt id even if the stage is the first one.

rxin · 2014-07-12T06:19:32Z

@tsudukim The concept of TaskSet should be internal to Spark. Users shouldn't have to aware of task set. Users should only care about stage + attempt.

tsudukim · 2014-07-12T06:41:03Z

@rxin OK, thanks. Then attempt id is still required in the web ui for users to know stage + attempt. Have I got that right?

rxin · 2014-07-12T06:45:37Z

Yup - but let's avoid exposing the concept of TaskSet to users in the UI. That's only for internal engineering.

tsudukim · 2014-07-14T21:37:26Z

I'm wondering how to show it. I gave it a shot. Is it smart?

kayousterhout · 2014-07-16T00:14:16Z

core/src/main/scala/org/apache/spark/util/JsonProtocol.scala

@@ -478,6 +479,7 @@ private[spark] object JsonProtocol {

  def stageInfoFromJson(json: JValue): StageInfo = {
    val stageId = (json \ "Stage ID").extract[Int]
+    val attemptId = (json \ "Attempt ID").extract[Int]


Is this backwards compatible?

Ah, no it's not. How about this?

val attemptId = (json \ "Attempt ID").extractOpt[Int].getOrElse(0)

Could you add a backwards compatible test in JsonProtocolSuite?

pwendell · 2014-07-16T00:43:40Z

@tsudukim @kayousterhout so I think in general here, our handling of stage re-submissions is broken in the UI. For instance, I looked in the JobProgressListener and we index many things on StageId that might better be indexed on StageId, AttemptId. Also, we should probably give the AttemptId when start a task so that we understand which stage attempt to associated it with. I also don't understand exactly what happens when a stage gets re-attempted, do we send a "stage completed" event? It might be good to fix the way we deal with stage re-submissions to make this work better overall.

kayousterhout · 2014-07-16T00:46:19Z

@rxin is this something you've thought about in your various schedule refactoring things?

pwendell · 2014-07-16T00:52:34Z

@tsudukim I created a JIRA to deal with the broader issue. If you want to take that on as well, let me know: https://issues.apache.org/jira/browse/SPARK-2501 it might make sense to wrap it into this patch.

tsudukim · 2014-07-16T18:34:49Z

@pwendell I agree that there are many room for improvement about handling of stageId and attemptId. It might be better to break this problems into some sub-tasks. I think this patch should be one of them. Or did you mean we should fix all of this problem in one patch?

rxin · 2014-07-16T18:39:43Z

Let's hold off merging this one until we merge #1262. Then it will be easier to index the information based on stage + attempt.

tsudukim · 2014-07-16T18:46:45Z

@rxin OK. After that, I think I can make this patch better.

tsudukim · 2014-07-16T18:51:13Z

@rxin in #1262, can I expect the key of the stagedata in JobProgressListener become stageId + attemptId instead of stageId only?

Added attempt ID column into stage page of webUI. Added attemptId handling code into StageInfo, JsonProtocol. Modified DAGScheduler to identify stages whose stageId is same but attemptId is different. Modified testcode for stage attempt ID.

Modified format of stageId and attemtId. Modified to backward compatible style. Added backward compatibility test.

tsudukim · 2014-07-19T01:26:04Z

Modified PR as your comments. thank you!

rxin · 2014-07-23T09:15:52Z

It turned out much trickier than I thought to add attempt id. I submitted a PR here #1545

That PR already modifies the UI, since that's the only way I could test.

lianhuiwang · 2014-07-23T11:15:50Z

i think we can add jobid to stageTable/UI. because jobid is very useful when a application has many jobs.that can distinguish every job's stages.

tsudukim · 2014-07-23T22:25:27Z

@rxin Surely we can also fix them all in one patch. But it can be a little bit hard work to modify them compatibly in one patch so I just have thought to separate into several tasks and to make #1384 as first step by showing only attemptId to distinguish attempts.
You can take whichever is convenient for you.

tsudukim · 2014-07-23T22:27:11Z

@lianhuiwang It appears to be a different problem to SPARK-2298.
Is your aim same as this ticket?
https://issues.apache.org/jira/browse/SPARK-1362
If so, how about creating another PR to modify it?

lianhuiwang · 2014-07-25T13:30:28Z

@tsudukim yes,SPARK-2298 is that i want to. but i think a simple way is on this PR add a jobid column to stage table.it is very easy to achieve it.

pwendell · 2014-09-02T01:39:19Z

I think this was ultimately fixed by #1545 so we can close this issue. But feel free to open another PR if that one did not fix this.

kayousterhout reviewed Jul 16, 2014
View reviewed changes

tsudukim added 2 commits July 18, 2014 17:44

SPARK-2298 Show stage attempt in UI

e9d89bc

Added attempt ID column into stage page of webUI. Added attemptId handling code into StageInfo, JsonProtocol. Modified DAGScheduler to identify stages whose stageId is same but attemptId is different. Modified testcode for stage attempt ID.

SPARK-2298: Show stage attempt in UI -2nd

130d13f

Modified format of stageId and attemtId. Modified to backward compatible style. Added backward compatibility test.

asfgit closed this in 1f98add Sep 2, 2014

SPARK-2298: Show stage attempt in UI #1384

SPARK-2298: Show stage attempt in UI #1384

Uh oh!

Conversation

tsudukim commented Jul 12, 2014

Uh oh!

AmplabJenkins commented Jul 12, 2014

Uh oh!

tsudukim commented Jul 12, 2014

Uh oh!

pwendell commented Jul 12, 2014

Uh oh!

pwendell commented Jul 12, 2014

Uh oh!

tsudukim commented Jul 12, 2014

Uh oh!

andrewor14 commented Jul 12, 2014

Uh oh!

tsudukim commented Jul 12, 2014

Uh oh!

rxin commented Jul 12, 2014

Uh oh!

tsudukim commented Jul 12, 2014

Uh oh!

rxin commented Jul 12, 2014

Uh oh!

tsudukim commented Jul 14, 2014

Uh oh!

kayousterhout Jul 16, 2014

Choose a reason for hiding this comment

Uh oh!

tsudukim Jul 16, 2014

Choose a reason for hiding this comment

Uh oh!

andrewor14 Jul 16, 2014

Choose a reason for hiding this comment

Uh oh!

tsudukim Jul 16, 2014

Choose a reason for hiding this comment

Uh oh!

pwendell commented Jul 16, 2014

Uh oh!

kayousterhout commented Jul 16, 2014

Uh oh!

pwendell commented Jul 16, 2014

Uh oh!

tsudukim commented Jul 16, 2014

Uh oh!

rxin commented Jul 16, 2014

Uh oh!

tsudukim commented Jul 16, 2014

Uh oh!

tsudukim commented Jul 16, 2014

Uh oh!

tsudukim commented Jul 19, 2014

Uh oh!

rxin commented Jul 23, 2014

Uh oh!

lianhuiwang commented Jul 23, 2014

Uh oh!

tsudukim commented Jul 23, 2014

Uh oh!

tsudukim commented Jul 23, 2014

Uh oh!

lianhuiwang commented Jul 25, 2014

Uh oh!

pwendell commented Sep 2, 2014

Uh oh!

Uh oh!