[SPARK-16925] Master should call schedule() after all executor exit events, not only failures

[JoshRosen]

What changes were proposed in this pull request?

This patch fixes a bug in Spark's standalone Master which could cause applications to hang if tasks cause executors to exit with zero exit codes.

As an example of the bug, run

sc.parallelize(1 to 1, 1).foreachPartition { _ => System.exit(0) }

on a standalone cluster which has a single Spark application. This will cause all executors to die but those executors won't be replaced unless another Spark application or worker joins or leaves the cluster (or if an executor exits with a non-zero exit code). This behavior is caused by a bug in how the Master handles the ExecutorStateChanged event: the current implementation calls schedule() only if the executor exited with a non-zero exit code, so a task which causes a JVM to unexpectedly exit "cleanly" will skip the schedule() call.

This patch addresses this by modifying the ExecutorStateChanged to always unconditionally call schedule(). This should be safe because it should always be safe to call schedule(); adding extra schedule() calls can only affect performance and should not introduce correctness bugs.

How was this patch tested?

I added a regression test in DistributedSuite.

[zsxwing]

LGTM

[SparkQA]

Test build #63286 has finished for PR 14510 at commit c567a7e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[JoshRosen]

I'm going to merge this to master, branch-2.0, and branch-1.6. I have a followup patch to add configuration options for controlling the "remove application that has experienced too many back-to-back executor failures" code path, which I'll submit tomorrow.