Skip to content

[SPARK-10918] [CORE] Prevent task failed for executor kill by driver #8975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

[SPARK-10918] [CORE] Prevent task failed for executor kill by driver #8975

wants to merge 4 commits into from

Conversation

shenh062326
Copy link
Contributor

When dynamicAllocation is enabled, when a executor was idle timeout, it will be kill by driver, if a task offer to the executor at the same time, the task will failed due to executor lost.

  1. Before scheduler a task to a executor, check if it has been killed.
  2. Call executorManager.onExecutorBusy before scheduler task.

@SparkQA
Copy link

SparkQA commented Oct 5, 2015

Test build #43230 has finished for PR 8975 at commit 88c9c3e.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class LogicalRelation(

@SparkQA
Copy link

SparkQA commented Oct 5, 2015

Test build #43235 has finished for PR 8975 at commit 18cbf73.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 19, 2015

Test build #43915 has finished for PR 8975 at commit 6215f52.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 19, 2015

Test build #43916 has finished for PR 8975 at commit de47fb6.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

@shenh062326 I've closed the issue because it duplicates SPARK-9552. Let's also close this PR because I believe #7888 is a better approach.

@vanzin
Copy link
Contributor

vanzin commented Dec 5, 2015

@shenh062326 could you close this PR?

@asfgit asfgit closed this in ce5fd40 Dec 17, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants