Skip to content

Possibility to define implied nodes for hierarchical test engine #3189

Open

Description

While trying to come up with a fix for spockframework/spock#1593, I came to the conclusion that it is probably not possible with manageable effort within the engine, but should be done in JUnit platform, where it imho belongs anyway.

The problem is, the following.
Imagine a test class with three test methods a, b, and c.
They are configured to must run in order and also to run c you have to run a and b first always, and to run b you have to run a first always.
(That's basically the logic of @Stepwise on the test class in Spock)

With Spock 1 which was a JUnit 4 runner this worked perfectly well.
But with Spock 2 which is a JUnit platform engine this does not work anymore.

The problem is, that typically (I checked Gradle, Maven, and IntelliJ) a PostDiscoveryFilter is used to match test methods, as the tools usually support some pattern syntax for the method name.

Due to that, the post discovery filter throws out the methods a and b as only c was configured to run.
But to correctly run c you have to run a and b first.

I played with different options, but did not yet find a viable solution to solve this properly within the engine, itself.
For example using the DynamicTestExecutor given to the execute method of the node for c does not work properly, as then first the setup / "before" method for c is run, then the setup method for b, then for a, then a and b are executed and finally c. This makes problems if you for example work with shared state between the tests. Besides that you then cannot configure needed resources for a or b as those are not supported on dynamically added tests.

So what I'm after and what other test engines might also find useful, is a way to declare / configure that node c implies nodes b and a and that node b implies a.
As a result, if after post discovery filtering only c would be left now, it should then also contain a and b.
If after post discovery filtering only b would be left, it should also contain a.


An alternative option might maybe be to have some possibility for a PostDiscoveryFilteringPostProcessor, so that the discovery result could again be amended after the post discovery filtering, so that the dependencies can be added back into the tree as needed.


If you have better ideas how to implement it or how I could do it within the engine properly, don't hesitate to tell me. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions