Draft support for cancelling test execution #4709

marcphilipp · 2025-07-01T15:17:35Z

Overview

Introduce CancellationToken interface and default implementation
Add Launcher.execute(TestPlan, CancellationToken, ...) method
Add ExecutionRequest.getCancellationToken() method
Implement support for cancellation for engines extending
HierarchicalTestEngine

Eventually resolves #1880.

I hereby agree to the terms of the JUnit Contributor License Agreement.

Definition of Done

There are no TODOs left in the code
Method preconditions are checked and documented in the method's Javadoc
Coding conventions (e.g. for logging) have been followed
Change is covered by automated tests including corner cases, errors, and exception handling
Public API has Javadoc and @API annotations
Change is documented in the User Guide and Release Notes

marcphilipp · 2025-07-01T15:20:30Z

junit-platform-launcher/src/main/java/org/junit/platform/launcher/Launcher.java

+	 * @since 6.0
+	 */
+	@API(status = EXPERIMENTAL, since = "6.0")
+	void execute(TestPlan testPlan, CancellationToken cancellationToken, TestExecutionListener... listeners);


New API for build tools and IDEs

marcphilipp · 2025-07-01T15:20:41Z

junit-platform-engine/src/main/java/org/junit/platform/engine/ExecutionRequest.java

@@ -171,4 +174,8 @@ public NamespacedHierarchicalStore<Namespace> getStore() {
 			"No NamespacedHierarchicalStore was configured for this request");
 	}

+	public CancellationToken getCancellationToken() {


New API for TestEngine implementors

marcphilipp · 2025-07-01T15:21:14Z

...atform-engine/src/main/java/org/junit/platform/engine/support/hierarchical/NodeTestTask.java

+	private SkipResult checkWhetherSkipped() throws Exception {
+		return taskContext.cancellationToken().isCancellationRequested() //
+				? CANCELLED_SKIP_RESULT //
+				: node.shouldBeSkipped(requiredContext());


Support for engines extending HierarchicalTestEngine

For Cucumber this means that cancellation will work out of the box. 👍

marcphilipp · 2025-07-01T15:23:17Z

...orm-launcher/src/main/java/org/junit/platform/launcher/core/EngineExecutionOrchestrator.java

+		if (cancellationToken.isCancellationRequested()) {
+			listener.executionSkipped(engineDescriptor, "Cancellation of execution requested");
+			return;
+		}


Skip entire engine if cancellation has already been requested. This avoid the overhead of initializing an engine's execution only to skip its root node. Moreover, this is at least a stop-gap solution for engines not supporting cancellation at all.

marcphilipp · 2025-07-01T15:24:12Z

jupiter-tests/src/test/java/org/junit/jupiter/engine/ExecutionCancellationTests.java

+		results.allEvents().assertEventsMatchExactly( //
+			event(engine(), started()), //
+			event(container(testClass), started()), //
+			event(test("first"), started()), //
+			event(test("first"), reportEntry(Map.of("cancelled", "true"))), //
+			event(test("first"), finishedSuccessfully()), //
+			event(test("second"), skippedWithReason("Test execution cancelled")), //
+			event(container(testClass), finishedSuccessfully()), //
+			event(engine(), finishedSuccessfully()));


This test demonstrates what happens if execution is cancelled during execution of the first test. The second test is reported as skipped and all later test classes would be as well.

* Introduce `CancellationToken` interface and default implementation * Add `Launcher.execute(TestPlan, CancellationToken, ...)` method * Add `ExecutionRequest.getCancellationToken()` method * Implement support for cancellation for engines extending `HierarchicalTestEngine` Eventually resolves #1880.

mpkorstanje · 2025-07-03T11:46:01Z

junit-platform-engine/src/main/java/org/junit/platform/engine/ExecutionRequest.java

@@ -171,4 +174,8 @@ public NamespacedHierarchicalStore<Namespace> getStore() {
 			"No NamespacedHierarchicalStore was configured for this request");
 	}

+	public CancellationToken getCancellationToken() {
+		return Preconditions.notNull(this.cancellationToken, "No CancellationToken was configured for this request");


I had some difficulty inferring from what code how I could safely use the cancellation token. Instead of making the cancellationToken nullable, I would expect a default implementation to be provided in the deprecated constructor.

I think we can instead return a default one here since it has a sensible default.

mpkorstanje · 2025-07-03T14:58:34Z

junit-platform-suite-engine/src/main/java/org/junit/platform/suite/engine/SuiteTestEngine.java

@@ -73,7 +73,7 @@ public void execute(ExecutionRequest request) {
 		suiteEngineDescriptor.getChildren()
 				.stream()
 				.map(SuiteTestDescriptor.class::cast)
-				.forEach(suiteTestDescriptor -> suiteTestDescriptor.execute(engineExecutionListener, requestLevelStore));
+				.forEach(suiteTestDescriptor -> suiteTestDescriptor.execute(engineExecutionListener, requestLevelStore, request.getCancellationToken()));


This will fail if a request was created with the deprecated constructor.

ExecutionRequest really should be created by anyone outside of the Platform.

Since many tools; for example, Maven Surefire; do not report events for engine-level descriptors, reporting their children as skipped increases compatibility with such tools.

The new `Launcher.execute(LauncherExecutionRequest)` method paves the road for adding additional parameters to test execution without having to add additional overloads of `execute()`. For example, for #1880 we will likely need to introduce a `CancellationToken` (as drafted in #4709). Instead of having to add two new overloads, such changes will then be easy because only `LauncherExecutionRequest` will need to be changed. `LauncherExecutionRequestBuilder` provides a fluent API for constructing execution requests starting either with a `TestPlan` or a `LauncherDiscoveryRequest`. In addition, `LauncherDiscoveryRequestBuilder.forExecution()` is a convenience method to create an execution request from a discovery request.

marcphilipp · 2025-07-07T07:47:38Z

Superseded by #4728 and #4725.

The new `Launcher.execute(LauncherExecutionRequest)` method paves the road for adding additional parameters to test execution without having to add additional overloads of `execute()`. For example, for #1880 we will likely need to introduce a `CancellationToken` (as drafted in #4709). Instead of having to add two new overloads, such changes will then be easy because only `LauncherExecutionRequest` will need to be changed. `LauncherExecutionRequestBuilder` provides a fluent API for constructing execution requests starting either with a `TestPlan` or a `LauncherDiscoveryRequest`. In addition, `LauncherDiscoveryRequestBuilder.forExecution()` is a convenience method to create an execution request from a discovery request.

marcphilipp self-assigned this Jul 1, 2025

github-actions bot added component: Platform theme: execution type: enhancement labels Jul 1, 2025

marcphilipp mentioned this pull request Jul 1, 2025

Introduce support for cancelling a running execution to the Launcher API #1880

Closed

marcphilipp commented Jul 1, 2025

View reviewed changes

marcphilipp force-pushed the marc/1880-execution-cancellation branch from 8ec7807 to 1172d71 Compare July 1, 2025 15:59

Use RunNotifier rather than JUnitCore

1b9086f

marcphilipp force-pushed the marc/1880-execution-cancellation branch from 8eb0df7 to 45ad996 Compare July 3, 2025 06:38

Add overload to execute a discovery request with a cancellation token

617b66e

marcphilipp force-pushed the marc/1880-execution-cancellation branch from 45ad996 to 617b66e Compare July 3, 2025 06:39

mpkorstanje reviewed Jul 3, 2025

View reviewed changes

Report children of engine descriptors as skipped

c02615e

Since many tools; for example, Maven Surefire; do not report events for engine-level descriptors, reporting their children as skipped increases compatibility with such tools.

marcphilipp mentioned this pull request Jul 5, 2025

Introduce LauncherExecutionRequest #4724

Merged

6 tasks

marcphilipp closed this Jul 7, 2025

marcphilipp deleted the marc/1880-execution-cancellation branch July 7, 2025 07:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Draft support for cancelling test execution #4709

Draft support for cancelling test execution #4709

Uh oh!

marcphilipp commented Jul 1, 2025

Uh oh!

marcphilipp Jul 1, 2025 •

edited

Loading

Uh oh!

marcphilipp Jul 1, 2025 •

edited

Loading

Uh oh!

marcphilipp Jul 1, 2025

Uh oh!

mpkorstanje Jul 3, 2025

Uh oh!

marcphilipp Jul 1, 2025

Uh oh!

marcphilipp Jul 1, 2025

Uh oh!

mpkorstanje Jul 3, 2025 •

edited

Loading

Uh oh!

marcphilipp Jul 6, 2025

Uh oh!

mpkorstanje Jul 3, 2025

Uh oh!

marcphilipp Jul 6, 2025

Uh oh!

marcphilipp commented Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

Draft support for cancelling test execution #4709

Draft support for cancelling test execution #4709

Uh oh!

Conversation

marcphilipp commented Jul 1, 2025

Overview

Definition of Done

Uh oh!

marcphilipp Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcphilipp Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcphilipp Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

marcphilipp Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

marcphilipp Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcphilipp Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

marcphilipp Jul 6, 2025

Choose a reason for hiding this comment

Uh oh!

marcphilipp commented Jul 7, 2025

Uh oh!

Uh oh!

marcphilipp Jul 1, 2025 •

edited

Loading

marcphilipp Jul 1, 2025 •

edited

Loading

mpkorstanje Jul 3, 2025 •

edited

Loading