How to detect directories recursively with @Suite?

[haleyk10198]

Premises:

We are migrating a JUnit4 test base to JUnit5. One of the patterns that the code base have is that tests are split to directories by their type (unit/integration etc.). An advantage we get from this is that we can better monitor test status, e.g. it's easier to specify CI to ignore smoke test failure. The grouping of these test suites by type is managed by a custom JUnit4 test runner and a crawler class that outputs the list of all directories recursively.

Issue:

JUnit5 @SelectDirectory does not crawl recursively. We cannot pass our crawler's output (list of all directories) to the annotation neither because its not evaluated at compile time. Also, to our knowledge there is no interface that is synonymous to customized JUnit4 runner / JUnit5 Suite Selector without writing our own test engine.

An alternative we have is to move the crawling to build script ( maven / gradle ), but we'd prefer keeping the test specifications in the code base.

TL;DR: How to pass a hierarchy to @suite scope?

[sbrannen]

For starters, @SelectDirectories (not @SelectDirectory) is not applicable for JUnit 4 or JUnit Jupiter test classes. That is only supported by a TestEngine that allows you to select directories in the file system.

How to pass a hierarchy to @suite scope?

You cannot pass any sort of hierarchy.

However, you can select classes by their names, packages, and/or tags.

One of the patterns that the code base have is that tests are split to directories by their type (unit/integration etc.).

How do you infer their "type"?

If it's based on the packages, use @SelectPackages/@ExcludePackages.

If it's based on their names, use @IncludeClassNamePatterns/@ExcludeClassNamePatterns.

If it's based on tags, use @IncludeTags/@ExcludeTags.

And you can of course use any combination of the aforementioned annotations.

[haleyk10198]

How do you infer their "type"?

The code is organized in [ "src/main", "src/test-unit", "src/test-integration", "src/test-${type}" ]

The package annotation is not feasible here, as a class Bar under package Foo "/src/main/Foo/BarTest" is intended to have different tests split under the same package Foo but different folders: "src/test-unit/Foo/BarUnitTest" and "src/test-integration/Foo/BarIntegrationTest"

I tried to convince my team to use ClassNamePatterns but was rejected because one could easily add a test without realizing the need of wiring it to the suite. Tags shares the same concern.

No combination replicates the crawling logic here.

[sbrannen]

I tried to convince my team to use ClassNamePatterns but was rejected because one could easily add a test without realizing the need of wiring it to the suite.

Well, you could create Checkstyle rules that enforce the naming convention per source folder.