[Flaky Test] tez unit test RssShuffleTest

[summaryzb]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the flaky test

[INFO] Running org.apache.tez.runtime.library.common.shuffle.orderedgrouped.RssShuffleTest
Error:  Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.134 s <<< FAILURE! - in org.apache.tez.runtime.library.common.shuffle.orderedgrouped.RssShuffleTest
Error:  testSchedulerTerminatesOnException  Time elapsed: 0.074 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <false> but was: <true>
	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
	at org.junit.jupiter.api.AssertFalse.assertFalse(AssertFalse.java:40)
	at org.junit.jupiter.api.AssertFalse.assertFalse(AssertFalse.java:35)
	at org.junit.jupiter.api.Assertions.assertFalse(Assertions.java:227)
	at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.RssShuffleTest.testSchedulerTerminatesOnException(RssShuffleTest.java:93)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)

Actions URL

https://github.com/apache/incubator-uniffle/actions/runs/6607344077/job/17944613341?pr=1253

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[summaryzb]

@lifeSo please take a look, appreciated

[zhengchenyu]

I also encountered this problem and prepare to fix it.

shuffle.run -> submit runnable to executor.
The runnable will call cleanupShuffleScheduler -> rssScheduler.close -> sShutdown.getAndSet(true)
So this is a random error.

If sleep 3s before 'assertFalse(scheduler.isShutdown())'. It is easy to reproduce this error.