TestEngines not found by ServiceLoader on Java 9

[sormuras]

Overview

This issue was discovered in conjunction with tests on Java 9; however, the issue is likely applicable on Java 8 as well.

The following pseudo-code, when run inside a Java 9 JShell session...

println("ServiceLoader.load(TestEngine.class)");
println("  loader = " + Thread.currentThread().getContextClassLoader());
ServiceLoader.load(TestEngine.class).forEach(System.out::println);

println("ServiceLoader.load(TestEngine.class, getClass().getClassLoader())");
println("  loader = " + getClass().getClassLoader());
ServiceLoader.load(TestEngine.class, getClass().getClassLoader()).forEach(System.out::println);

println("ServiceLoader.load(TestEngine.class, TestEngine.class.getClassLoader())");
println("  loader = " + TestEngine.class.getClassLoader());
ServiceLoader.load(TestEngine.class, TestEngine.class.getClassLoader()).forEach(System.out::println);

yields the following :

ServiceLoader.load(TestEngine.class)
  loader = jdk.jshell.execution.DefaultLoaderDelegate$RemoteClassLoader@4b952a2d
-

ServiceLoader.load(TestEngine.class, getClass().getClassLoader())
  loader = jdk.internal.loader.Loader@42607a4f
* org.junit.jupiter.engine.JupiterTestEngine@fa36558
* org.junit.vintage.engine.VintageTestEngine@672872e1

ServiceLoader.load(TestEngine.class, TestEngine.class.getClassLoader())
  loader = jdk.internal.loader.Loader@42607a4f
* org.junit.jupiter.engine.JupiterTestEngine@484970b0
* org.junit.vintage.engine.VintageTestEngine@4470f8a6

The problem leads to org.junit.platform.commons.util.PreconditionViolationException: Cannot create Launcher without at least one TestEngine; consider adding an engine implementation JAR to the classpath messages and was first encountered at forax/pro#23 (comment)

Work-around

• A work-around is to use getClass().getClassLoader() as the result of ClassLoaderUtils.getDefaultClassLoader().
• The user (tool) has to set context classloader to the one that contains the engines: External Java command line tool "launcher" feature forax/pro#23 (comment)

Deliverables

• Investigate a safer/better way to load available engines (and other objects) at runtime.
  • see also: Revise ClassLoader lookups across the platform #806

[sbrannen]

I think the bug you've reported is actually a symptom of a larger problem, namely class loading in general.

We (or at least I) always assumed that our use of ClassLoaderUtils.getDefaultClassLoader() would only get us so far (i.e., for the most common class loading scenarios).

My proposal is to open a new issue dedicated to the topic of class loading and refer to this issue as a concrete failing scenario. The new issue would serve as an umbrella issue and include all of the use cases you list in the Possibly affected section.

If we take inspiration from the Spring Framework, we can see that org.springframework.util.ClassUtils.forName(String, ClassLoader) accepts a user-supplied ClassLoader and falls back to the "default ClassLoader" only if the supplied ClassLoader is null. For loading services, Spring does not use Java's ServiceLoader mechanism but rather its own SpringFactoriesLoader which performs the same task as Java's ServiceLoader but with explicit control over which ClassLoader is used. Note that SpringFactoriesLoader delegates to org.springframework.util.ClassUtils.forName(String, ClassLoader) to actually load the service class.

I'm not saying that JUnit necessarily needs the flexible support of SpringFactoriesLoader, but we will likely have to supply a custom ClassLoader in such cases instead of relying on the "default ClassLoader" which currently is typically the thread context ClassLoader (TCCL).

Thus, as you mentioned supplying getClass().getClassLoader() is likely a viable option. However, I would not change the implementation of ClassLoaderUtils.getDefaultClassLoader(). Rather, I would simply use it less often and then only as a fallback.

FWIW, we actually did have the foresight to create org.junit.platform.commons.util.ReflectionUtils.loadClass(String, ClassLoader), but... we never supply it anything other than the default ClassLoader. 😉

[sbrannen]

FYI: the Possibly affected section has been moved to #806.

[sormuras]

Using the TCCL solved the issue.