[SPARK-3547]Using a special exit code instead of 1 to represent ClassNotFoundExcepti...

[WangTaoTheTonic]

...on

As improvement of #1944, we should use more special exit code to represent ClassNotFoundException.

[SparkQA]

QA tests have started for PR 2421 at commit fbb232f.

• This patch merges cleanly.

[WangTaoTheTonic]

@liancheng What do you think?

[SparkQA]

Tests timed out after a configured wait of 120m.

[liancheng]

Hmm, considering that SparkSubmit can be used to start any user application, which may call System.exit(Int) at any time with an arbitrary integer, it can a good idea to pick a more special exit code. LGTM.

[liancheng]

test this please

[sarutak]

Exit code should be between 0 - 255. It's Bash's specification.

[liancheng]

Ah, good point! The integer is converted to an unsigned byte under bash. And for 1024, the converted value is actually 0, which is definitely not acceptable... Maybe some prime number like 251 instead?

[vanzin]

Please keep error codes < 128. From http://en.wikipedia.org/wiki/Exit_status:

When a command terminates on a fatal signal whose number is N, Bash uses the value 128+N as the exit status.

[liancheng]

Thanks for the reference! Good to know, didn't know these specific rules before.

[WangTaoTheTonic]

Use 127 instead, it is the biggest prime number in those less than 128.

How about it, guys?

[SparkQA]

QA tests have started for PR 2421 at commit a2d6465.

• This patch merges cleanly.

[liancheng]

@WangTaoTheTonic According to the wiki page @vanzin pointed out, values above 125 are used by bash for special purposes. Since the purpose of this PR is to reduce the possibility of exit code collision, just choose any value that is not too common and doesn't violate bash rules, not necessary to be prime :)

[WangTaoTheTonic]

Sorry for not noticing "If a command is not found, the child process created to execute it returns a status of 127. If a command is found but is not executable, the return status is 126".

Now using 101.

[SparkQA]

QA tests have started for PR 2421 at commit d6ae559.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2421 at commit a2d6465.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

QA tests have finished for PR 2421 at commit d6ae559.

• This patch fails unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WangTaoTheTonic]

Gosh, the test failed. I looked "block generator throttling" in NetworkReceiverSuite.scala but couldn't see why.

[pwendell]

Jenkins, test this please.

[SparkQA]

QA tests have started for PR 2421 at commit d6ae559.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2421 at commit d6ae559.

• This patch fails unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

QA tests have started for PR 2421 at commit 645a22a.

• This patch merges cleanly.

[sarutak]

@WangTaoTheTonic You can let Jenkins works by saying like "retest this please" or "test this please" without extra push.

[SparkQA]

QA tests have started for PR 2421 at commit 645a22a.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2421 at commit 645a22a.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WangTaoTheTonic]

@sarutak Thanks I see, thought only commiters can do it this way.

[SparkQA]

QA tests have finished for PR 2421 at commit 645a22a.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@sarutak Actually Jenkins only listens to a limited group of people, and sometimes he even ignores this group for unknown reasons. Lots of work led by Josh had been done in the past weeks to make Jenkins happy, I think he'll be much more friendly and responsive soon :)

[sarutak]

@liancheng Ah, I saw sometimes Jenkins ignores us... but recently he is friendly :D