Description
Andrew Toolan opened SPR-14112 and commented
Background:
I am trying to run JUnit integration tests in parallel and repeatedly using the SpringClassRule and SpringMethodRule. To do this I am using a custom runner which executes the tests in parallel and using a repeat annotation to run each test repeatedly. However I have noticed that after multiple iterations the tests will begin failing. One test always appears to succeed. The resulting stack trace seems to indicate that the Application Context has been changed. The tests do not make use of the @DirtiesContext annotation however some of the errors received appear to complain about the test context being null. I am attaching a few examples of the stack traces, below is an example:
java.lang.IllegalArgumentException: The test method of the supplied TestContext must not be null
at org.springframework.util.Assert.notNull(Assert.java:115)
at org.springframework.test.context.support.AbstractDirtiesContextTestExecutionListener.beforeOrAfterTestMethod(AbstractDirtiesContextTestExecutionListener.java:95)
at org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener.beforeTestMethod(DirtiesContextBeforeModesTestExecutionListener.java:95)
at org.springframework.test.context.TestContextManager.beforeTestMethod(TestContextManager.java:265)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
at org.springframework.test.context.junit4.statements.RunPrepareTestInstanceCallbacks.evaluate(RunPrepareTestInstanceCallbacks.java:64)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
at org.springframework.test.context.junit4.statements.SpringFailOnTimeout.evaluate(SpringFailOnTimeout.java:87)
at org.springframework.test.context.junit4.statements.ProfileValueChecker.evaluate(ProfileValueChecker.java:103)
at com.ibm.toscana.util.junit.repeat.RunRepeatedly.evaluateStatement(RunRepeatedly.java:55)
at com.ibm.toscana.util.junit.repeat.RunRepeatedly.evaluate(RunRepeatedly.java:34)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.experimental.theories.Theories$TheoryAnchor$1$1.evaluate(Theories.java:232)
at org.junit.experimental.theories.Theories$TheoryAnchor.runWithCompleteAssignment(Theories.java:218)
at org.junit.experimental.theories.Theories$TheoryAnchor.runWithAssignment(Theories.java:204)
at com.ibm.toscana.util.junit.runner.ToscanaRunner$ParallelTheoryAnchor.runWithAssignment(ToscanaRunner.java:70)
at org.junit.experimental.theories.Theories$TheoryAnchor.evaluate(Theories.java:187)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1397)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:300)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:168)
I have tried changing from the default Spring Repeat annotation to another implementation but I am seeing the same behavior. All tests are within the same test class. I was wondering if there was any reason why using SpringClassRule and SpringMethodRule in parallel might cause problems?
I was hoping to try run the tests without the SpringMethodRule being applied as I do not need to leverage any of Springs Method level annotations but I noticed that there is an explicit dependency between them.
I am using spring-boot version 1.3.2.RELEASE and JUnit 4.12.
Attachments:
- failed_tests_stacktraces.txt (10.52 kB)
Issue Links:
- Allow for concurrent test execution in the TestContext framework [SPR-5863] #10532 Allow for concurrent test execution in the TestContext framework ("duplicates")