[SPARK-3984] [SPARK-3983] Fix incorrect scheduler delay and display task deserialization time in UI #2832

kayousterhout · 2014-10-17T01:11:09Z

This commit fixes the scheduler delay in the UI (which previously
included things that are not scheduler delay, like time to
deserialize the task and serialize the result), and also
adds information about time to deserialize tasks to the optional
additional metrics. Time to deserialize the task can be large relative
to task time for short jobs, and understanding when it is high can help
developers realize that they should try to reduce closure size (e.g, by including
less data in the task description).

cc @shivaram @etrain

SparkQA · 2014-10-17T01:14:44Z

QA tests have started for PR 2832 at commit 81fb86b.

This patch merges cleanly.

SparkQA · 2014-10-17T01:15:56Z

QA tests have finished for PR 2832 at commit 81fb86b.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-10-17T01:15:57Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21828/
Test FAILed.

andrewor14 · 2014-10-20T18:53:20Z

retest this please

SparkQA · 2014-10-20T18:59:54Z

QA tests have started for PR 2832 at commit 81fb86b.

This patch merges cleanly.

SparkQA · 2014-10-20T19:01:00Z

QA tests have finished for PR 2832 at commit 81fb86b.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-10-20T19:01:01Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21927/
Test FAILed.

shivaram · 2014-10-20T19:25:11Z

@kayousterhout - This is failing scalastyle checks -- Could you run style check locally ?

kayousterhout · 2014-10-20T22:37:25Z

I'm holding off on this until I finish https://issues.apache.org/jira/browse/SPARK-4016 due to the concern that otherwise these new metrics will add confusion for the average user.

JoshRosen · 2014-10-31T17:29:47Z

Now that I've merged #2867, this should be unblocked.

kayousterhout · 2014-10-31T18:23:02Z

Thanks @JoshRosen -- fixing this up now!

JoshRosen · 2014-10-31T18:43:12Z

Just a head's up: I merged #3031, the "hide accumulators column when empty" patch, so it's likely to cause conflicts here; you might want to merge / rebase.

This commit fixes the scheduler delay in the UI (which previously included things that are not scheduler delay, like time to deserialize the task and serialize the result), and also adds finer-grained information to the summary table for each stage about task launch overhead (which is useful for debugging performance of short jobs, where the overhead is not-insignificant).

kayousterhout · 2014-11-01T00:26:07Z

Updated this so the two new metrics are hideable!

SparkQA · 2014-11-01T00:27:26Z

Test build #22666 has started for PR 2832 at commit 335be4b.

This patch merges cleanly.

SparkQA · 2014-11-01T01:41:14Z

Test build #22666 has finished for PR 2832 at commit 335be4b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-01T01:41:17Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22666/
Test PASSed.

kayousterhout · 2014-11-01T23:15:03Z

@JoshRosen @andrewor14 does one of you have time to take a look at this?

SparkQA · 2014-11-01T23:20:21Z

Test build #22710 has started for PR 2832 at commit 1f13afe.

This patch merges cleanly.

SparkQA · 2014-11-02T00:38:30Z

Test build #22710 has finished for PR 2832 at commit 1f13afe.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-02T00:38:33Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22710/
Test PASSed.

SparkQA · 2014-11-05T07:00:17Z

Test build #22920 has started for PR 2832 at commit 531575d.

This patch merges cleanly.

SparkQA · 2014-11-05T08:25:28Z

Test build #22920 has finished for PR 2832 at commit 531575d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-05T08:25:32Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22920/
Test PASSed.

kayousterhout · 2014-11-05T18:20:03Z

@pwendell as per our discussion, I changed this to eliminate the additional metric about task launch, so now this change just fixes the scheduler delay to be correct, and show task deserialization time in the UI. Does this look OK?

pwendell · 2014-11-05T19:23:27Z

LGTM!

SparkQA · 2014-11-05T19:52:36Z

Test build #22946 has started for PR 2832 at commit 0c1398e.

This patch merges cleanly.

SparkQA · 2014-11-05T20:58:20Z

Test build #22946 has finished for PR 2832 at commit 0c1398e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-05T20:58:23Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22946/
Test FAILed.

kayousterhout · 2014-11-05T21:00:02Z

Jenkins, retest this please

SparkQA · 2014-11-05T21:02:25Z

Test build #22949 has started for PR 2832 at commit 0c1398e.

This patch merges cleanly.

SparkQA · 2014-11-05T22:27:08Z

Test build #22949 has finished for PR 2832 at commit 0c1398e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-05T22:27:11Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22949/
Test PASSed.

andrewor14 · 2014-11-05T23:18:02Z

LGTM too

…ask deserialization time in UI This commit fixes the scheduler delay in the UI (which previously included things that are not scheduler delay, like time to deserialize the task and serialize the result), and also adds information about time to deserialize tasks to the optional additional metrics. Time to deserialize the task can be large relative to task time for short jobs, and understanding when it is high can help developers realize that they should try to reduce closure size (e.g, by including less data in the task description). cc shivaram etrain Author: Kay Ousterhout <kayousterhout@gmail.com> Closes #2832 from kayousterhout/SPARK-3983 and squashes the following commits: 0c1398e [Kay Ousterhout] Fixed ordering 531575d [Kay Ousterhout] Removed executor launch time 1f13afe [Kay Ousterhout] Minor spacing fixes 335be4b [Kay Ousterhout] Made metrics hideable 5bc3cba [Kay Ousterhout] [SPARK-3984] [SPARK-3983] Improve UI task metrics. (cherry picked from commit a46497e) Signed-off-by: Kay Ousterhout <kayousterhout@gmail.com>

kayousterhout · 2014-11-05T23:34:04Z

Thanks for looking at this @andrewor14 and @pwendell ! I've merged it into master and 1.2.

kayousterhout added 2 commits October 31, 2014 14:35

Made metrics hideable

335be4b

kayousterhout force-pushed the SPARK-3983 branch from 81fb86b to 335be4b Compare November 1, 2014 00:22

Minor spacing fixes

1f13afe

Removed executor launch time

531575d

kayousterhout changed the title ~~[SPARK-3984] [SPARK-3983] Improve UI task metrics.~~ [SPARK-3984] [SPARK-3983] Fix incorrect scheduler delay and display task deserialization time in UI Nov 5, 2014

Fixed ordering

0c1398e

asfgit closed this in a46497e Nov 5, 2014

[SPARK-3984] [SPARK-3983] Fix incorrect scheduler delay and display task deserialization time in UI #2832

[SPARK-3984] [SPARK-3983] Fix incorrect scheduler delay and display task deserialization time in UI #2832

Uh oh!

Conversation

kayousterhout commented Oct 17, 2014

Uh oh!

SparkQA commented Oct 17, 2014

Uh oh!

SparkQA commented Oct 17, 2014

Uh oh!

AmplabJenkins commented Oct 17, 2014

Uh oh!

andrewor14 commented Oct 20, 2014

Uh oh!

SparkQA commented Oct 20, 2014

Uh oh!

SparkQA commented Oct 20, 2014

Uh oh!

AmplabJenkins commented Oct 20, 2014

Uh oh!

shivaram commented Oct 20, 2014

Uh oh!

kayousterhout commented Oct 20, 2014

Uh oh!

JoshRosen commented Oct 31, 2014

Uh oh!

kayousterhout commented Oct 31, 2014

Uh oh!

JoshRosen commented Oct 31, 2014

Uh oh!

kayousterhout commented Nov 1, 2014

Uh oh!

SparkQA commented Nov 1, 2014

Uh oh!

SparkQA commented Nov 1, 2014

Uh oh!

AmplabJenkins commented Nov 1, 2014

Uh oh!

kayousterhout commented Nov 1, 2014

Uh oh!

SparkQA commented Nov 1, 2014

Uh oh!

SparkQA commented Nov 2, 2014

Uh oh!

AmplabJenkins commented Nov 2, 2014

Uh oh!

SparkQA commented Nov 5, 2014

Uh oh!

SparkQA commented Nov 5, 2014

Uh oh!

AmplabJenkins commented Nov 5, 2014

Uh oh!

kayousterhout commented Nov 5, 2014

Uh oh!

pwendell commented Nov 5, 2014

Uh oh!

SparkQA commented Nov 5, 2014

Uh oh!

SparkQA commented Nov 5, 2014

Uh oh!

AmplabJenkins commented Nov 5, 2014

Uh oh!

kayousterhout commented Nov 5, 2014

Uh oh!

SparkQA commented Nov 5, 2014

Uh oh!

SparkQA commented Nov 5, 2014

Uh oh!

AmplabJenkins commented Nov 5, 2014

Uh oh!

andrewor14 commented Nov 5, 2014

Uh oh!

kayousterhout commented Nov 5, 2014

Uh oh!

Uh oh!