[#2725] feat(spark): Introduce optional fast-switch and ignore retry-count checking for stale assignment

[zuston]

What changes were proposed in this pull request?

1. Introduce option for stale assignment fast-switch. After having this, we could better to inspect some bugs if this mechanism is caused
2. Ignore retry count checking for stale assignment to fix the multi server switch

Why are the changes needed?

for #2725

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests

[zuston]

cc @roryqi

[github-actions]

Test Results

 3 175 files  ±0   3 175 suites  ±0   6h 51m 25s ⏱️ -15s
 1 241 tests ±0   1 240 ✅ ±0   1 💤 ±0  0 ❌ ±0 
15 734 runs  ±0  15 719 ✅ ±0  15 💤 ±0  0 ❌ ±0 

Results for commit ff1b9e9. ± Comparison against base commit 1f809ed.

♻️ This comment has been updated with latest results.

[roryqi]

Is it a fallback mechanism?

[zuston]

Is it a fallback mechanism?

bug fix

[zuston]

Although the existing tests cover most scenarios in this PR, it’s difficult to simulate this specific case. After the PR #2727 introduces the ReassignExecutor, I will add dedicated tests for this PR. cc @roryqi

[roryqi]

Is it a fallback mechanism?

bug fix

Why do we need a config option if this is a bug fix?

[zuston]

Is it a fallback mechanism?

bug fix

Why do we need a config option if this is a bug fix?

This has been described in the description.

[roryqi]

1. inspect

The description says that this is a feature to find bugs easily instead of a bug.

[roryqi]

1. inspect

The description says that this is a feature to find bugs easily instead of a bug.

I got it. This pull request includes one feature and one fix.