Allow custom test runners to create their own TestClasses and customize the scanning of annotations.

[motlin]

This is to allow the creation of a test runner that looks for @test annotations on virtual extension methods in (Java 8) interfaces, as discussed on this closed pull-request.

You can see the test runner I built on these extensions here. It's a work in progress.

[dsaff]

I'd rather leave the field private, and have subclasses use getJavaClass(). Does that work?

[motlin]

Absolutely. I didn't notice the getJavaClass() method.

[dsaff]

Looks promising.

[motlin]

Thanks for the pointer. I could squash these two commits if you'd like.

[dsaff]

Yep, if you can squash, I'll pull. Thanks!

[motlin]

done

[dsaff]

Thanks!

[kcooney]

@motlin did your custom test runner extend from BlockJUnit4ClassRunner or something else? I want to make sure that #1449 doesn't break you.

[motlin]

@kcooney Thanks so much for checking. I'm having a bit of trouble following the new commit. If it helps, here is our custom test runner: https://github.com/motlin/eclipse-collections/tree/master/junit-trait-runner/src/main/java/org/eclipse/collections/impl/test/junit

The only customization in the runner is to override the implementation of TestClass. And TestClass is overridden to change the behavior of scanAnnotatedMembers, so that it scans interface (default) methods that are annotated and call addToAnnotationLists on those. If there's a newer way to achieve the same goal, I'm happy to upgrade.

[kcooney]

@motlin that will continue to work after I merge that pull.

Here's how someone would write that class in 4.13:

public class Java8Runner extends BlockJUnit4ClassRunner {

    /**
     * Creates a Java8Runner to run {@code testClass}.
     * Usually called via reflection.
     *
     * @throws InitializationError if the test class is malformed.
     */
    public Java8Runner(Class<?> testClass) throws InitializationError {
        super(new Java8TestClass(testClass));
    }
}

We will be deprecating ParentRunner.createTestClass() since it won't be called for subclasses of BlockJUnit4ClassRunner that use the constructor that takes in a TestClass. The new way is cleaner (and probably what we should have done to resolve your need in the first place),

[motlin]

@kcooney Sounds good, thanks.

I still feel like our test runner's behavior would be good default behavior in JUnit 4. I haven't tried to promote our test runner and I realize this style of testing isn't common. But there's also little downside. The extra reflection code won't take much time since most tests don't implement any interfaces.

We have a big and growing test suite here

One nice example that shows the benefit is UnmodifiableListTestCase which inherits the tests for unmodifiable collections and for lists.

The concrete tests that are classes are always small and usually implement one method which instantiates the collection under test, like UnmodifiableMutableListTest.

Many, many tests live at the top of our trait hierarchy in RichIterableTestCase.

[kcooney]

@motlin JUnit 5 will support defining tests in default methods.