Cannot select @TestTemplate or @TestFactory invocations by Unique ID

[noopur2507]

Overview

With reference to bug request 520923 in Eclipse, I am trying to rerun a single invocation of a parameterized test (and similarly, a repeated and a dynamic test) using its unique ID like this:

LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request().selectors(DiscoverySelectors.selectUniqueId(uniqueId)).build();

A new Launcher instance is created to discover the above request.

This method works for a normal test with ID:

[engine:junit-jupiter]/[class:test.FirstTest]/[method:myFirstTest(org.junit.jupiter.api.TestInfo)]

But it gives this error for a dynamic test:

org.junit.jupiter.engine.discovery.JavaElementsResolver resolveUniqueId
WARNING: Unique ID '[engine:junit-jupiter]/[class:test.dynamic.DynamicTestsDemo]/[test-factory:dynamicTestsFromIntStream()]/[dynamic-test:#2]' could not be resolved

or a parameterized test:

org.junit.jupiter.engine.discovery.JavaElementsResolver resolveUniqueId
WARNING: Unique ID '[engine:junit-jupiter]/[class:jb.Junit5SimpleParamTest]/[test-template:dummy(java.lang.String)]/[test-template-invocation:#1]' could not be resolved

In JUnit 4, it was done using org.junit.runner.Request.filterWith(Filter filter).

Question

What is the recommended way to do this in JUnit 5?

Related Issues

• JupiterTestEngine attempts to resolve unique IDs that belong to other engines #1026
• Status quo for selecting @TestTemplate or @TestFactory invocations by Unique ID #1031

Analysis

Given a Unique ID such as [engine:junit-jupiter]/[class:test.dynamic.DynamicTestsDemo]/[test-factory:dynamicTestsFromIntStream()]/[dynamic-test:#2],
JavaElementsResolver.resolveUniqueId(TestDescriptor, List<Segment>) returns true if all segments of the supplied Unique ID could be resolved.

So, in such a case, the test class and test factory method do in fact get resolved, but the dynamic test itself does not get resolved.

That means that the test class and test factory method will be executed in the test plan even though there is a warning (which actually only applies to the last segment of the Unique ID).

Proposals

1. Selection of any invocation of a test template or test factory results in selection of the test template or test factory as a container and consequently the selection of all invocations.
   • This appears to already be the case: see Analysis for details.
2. Selection of a specific invocation of a test template or test factory results in selection of only that specific invocation of the test template or test factory.
   • This could potentially be achieved via a special type of filter that is applied to the stream of dynamic tests or test template invocations.

Deliverables

• Ensure that an invocation of a @TestTemplate method can be selected by Unique ID.
  • This applies to @RepeatedTest and @ParameterizedTest as well as any test template method.
• Ensure that an invocation of a dynamic test supplied by a @TestFactory method can be selected by Unique ID.

[sbrannen]

There is no way to do that.

It's a bug.

I actually assumed that this bug existed but forgot to raise an issue, so.... thank you for raising the issue!!!

[sbrannen]

In general, there is no way to select a single invocation of a @TestTemplate or @TestFactory method by unique ID.

[sbrannen]

I have reworded the title of this issue accordingly.

[sbrannen]

Added Deliverables section

[sbrannen]

@junit-team/junit-lambda

I added the team discussion label to discuss "what" to select in such cases.

See my Proposal in Deliverables.

[sbrannen]

Moved Proposal to stand-alone section and added a deliverable for dynamic tests.

[sbrannen]

Update:

Now that I've read the original bug description in the Eclipse issue tracker, I realize that the user wants to be able to select a single dynamic test or invocation of a test template method.

@junit-team/junit-lambda, so we'll have to discuss the topic in detail to come up with a decision on what to do pre/post 5.0 GA.

[sbrannen]

Added 2nd, alternate proposal