throw SkipException in SuiteListener:onStart will hang start from 7.9.0

[sonwan2020]

We have lots of suites, and we use a centralized testng.xml with

Sometimes we want to skip some suites conditionally, so I add some logic in SuiteListener:onStart, and throw SkipException on some conditions.

Before release 7.8.0, this works.

But start from 7.9.0, there is a NPE error on thread, and the main job just hang there:

Exception in thread "TestNG-suites--1" java.lang.NullPointerException: Cannot invoke "org.testng.thread.IWorker.getTasks()" because "worker" is null
        at org.testng.internal.thread.graph.GraphOrchestrator.setStatus(GraphOrchestrator.java:119)
        at org.testng.internal.thread.graph.GraphOrchestrator.afterExecute(GraphOrchestrator.java:78)
        at org.testng.internal.thread.graph.TestNGFutureTask.done(TestNGFutureTask.java:34)
        at java.base/java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:381)
        at java.base/java.util.concurrent.FutureTask.setException(FutureTask.java:250)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:269)
        at org.testng.internal.thread.graph.TestNGFutureTask.run(TestNGFutureTask.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)

What is the suggested way to skip a suite conditionally?

I have append a simple project to reproduce the failures.
just run mvn test in the project.

• if I set the testng version to 7.8.0, the behavior is expected: suite 2 run and finished, suite 1 skipped.
• If I set the testng version to 7.9.0, suite 2 run and finished, NPE on suite 1 and the command just hang there.
• same NPE and hang behavior with the latest 7.10.1

test.zip

[sonwan2020]

I implemented the interface IAlterSuiteListener, and clear the tests conditionally.
And the result is expected.

[chandrakanthgunti]

hi Songbo Wang,

I am experiencing same issue while executing my automation scripts. Can you plz provide a sample code snippet or please provide more details how you resolved this issue?

Thanks in Advance.

[chandrakanthgunti]

hi Songbo Wang,

I am experiencing same issue while executing my automation scripts. Can you plz provide a sample code snippet or please provide more details how you resolved this issue?

Thanks in Advance.

[sonwan2020]

Hi @chandrakanthgunti,

I created a listener to implement interface IAlterSuiteListener
In the body of alter(List xmlSuites), I check the conditions, when I want to skip a suite, I just 'alter' the suite to be empty

public void alter(List<XmlSuite> xmlSuites) {
  for (XmlSuite xmlSuite : xmlSuites) {
      for (XmlSuite suite : xmlSuite.getChildSuites()) {
          if (<suite to be skipped>) {
              log.info(<msg>);
              suite.setTests(new ArrayList<>() {});
          }
      }
  }
}

[mfoo]

Hi all. I note that this is closed, presumably because there is a workaround, but I believe it still exists on 7.11.0 and should be fixed. Stack trace from Java 21.0.4:

Exception in thread "TestNG-test-Surefire test-1" java.lang.NullPointerException: Cannot invoke "org.testng.thread.IWorker.getTasks()" because "worker" is null
        at org.testng.internal.thread.graph.GraphOrchestrator.setStatus(GraphOrchestrator.java:119)
        at org.testng.internal.thread.graph.GraphOrchestrator.afterExecute(GraphOrchestrator.java:78)
        at org.testng.internal.thread.graph.TestNGFutureTask.done(TestNGFutureTask.java:34)
        at java.base/java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:434)
        at java.base/java.util.concurrent.FutureTask.setException(FutureTask.java:303)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:322)
        at org.testng.internal.thread.graph.TestNGFutureTask.run(TestNGFutureTask.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

Example use-case:

public interface ISomeFeatureTest extends WithEnvConfig {

    @BeforeMethod(alwaysRun = true)
    default void env_check() {
        if (!getEnvConfig().isSomeFeatureEnabled()) {
            throw new SkipException("Skipped until This Feature is deployed for this environment");
        }
    }
}

It results in the tests running forever.

We do this because the test suite could be run against various environments with various features enabled and we don't want to have to specify test groups, because that would require remembering what they are.

[juherr]

Related to #3238