Description
This might not be a bug and may be as intended, but kinda hit me by surprise as I don't see anything about it in the user guide or java docs. It appears that when you generate DynamicTests they are different from regular @Test methods in not generating separate instances of the test class containing them for each method. For example, the following code:
public class FooTest {
@Test
public void foo() {
System.out.println( this );
}
@TestFactory
public Stream< DynamicTest > testFactory() {
return Stream.of( "A", "B" )
.map( str -> DynamicTest.dynamicTest( "testFromFactory" + str + "()",
() -> System.out.println( str + " - " + this ) ) );
}
}might have sample output such as this:
Running org.littleclay.FooTest
org.littleclay.FooTest@5cc7c2a6
A - org.littleclay.FooTest@32d992b2
B - org.littleclay.FooTest@32d992b2
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec - in org.littleclay.FooTest
Note that the method foo has a separate instance of FooTest during it's execution but that both A and B tests share the same FooTest instance. It seems to me like this is a bit of a "gotcha" in using dynamic tests and think it should probably be spelled out more clearly if this is a necessary by-product of this type of test.
In a similar vein, it seems that the extension callbacks with dynamic tests have the same "method" which threw me a bit by surprise when following the examples regarding storing and retrieving state with extensions. Again, it makes sense with how I'm guessing these are generated, but if they can't be made more isolated, I definitely think a bit more information regarding life-cycles and possible contention points would be nice.
It is probably less of an issue if these types of dynamic tests can never be run in parallel, but I already see a few issues with my initial use case for them for creating parameterized tests if they cannot be (1) run in parallel and (2) be more isolated.