Limit max pool size for parallel execution via config param (#3044)

JUnit Jupiter (and The JUnit Platform) now support limiting the maximum
number of concurrently executing tests via the
`junit.jupiter.execution.parallel.config.fixed.max-pool-size`
configuration parameter.

With Java 9+ the `ForkJoinPool` used by JUnit can be configured with a
maximum pool size. While the number of concurrently executing tests may
exceed the configured parallelism when tests become blocked, it will
not exceed the maximum pool size.

Additionally, because the `ForkJoinPool` will by default reject tasks
that would exceed the maximum pool size the
`junit.jupiter.execution.parallel.config.fixed.saturate` property has
been added and will default to `true`. There appears to be no reason to
ever set this `false` but it is there should someone depend on the old
behavior.

These changes were intentionally not made to the `dynamic` strategy to
limit the scope of this pull request. While I can reasonably predict
what behavior users of the `fixed` strategy might expect, I can not say
the same about the `dynamic` strategy.

Fixes #3026.
Fixes #2545.
Fixes #1858.

Author: mpkorstanje

documentation/src/docs/asciidoc/release-notes/release-notes-5.10.0-M1.adoc

@@ -26,6 +26,8 @@ repository on GitHub.
 
 ==== New Features and Improvements
 
+* Support for limiting the `max-pool-size` for parallel execution via a configuration parameter
+
 * All utility methods from `ReflectionSupport` now have counterparts returning `Stream`
   instead of `List`.
 
@@ -38,10 +40,15 @@ repository on GitHub.
 
 ==== Deprecations and Breaking Changes
 
-* ❓
+* The `fixed` parallel execution strategy now allows the thread pool to be saturated by
+  default.
 
 ==== New Features and Improvements
 
+* New `junit.jupiter.execution.parallel.config.fixed.max-pool-size` configuration
+  parameter to set the maximum pool size.
+* New `junit.jupiter.execution.parallel.config.fixed.saturate` configuration
+  parameter to disable pool saturation.
 * New `ArgumentsAccessor.getInvocationIndex()` method that supplies the index of a
   `@ParameterizedTest` invocation.
 * `DisplayNameGenerator` methods are now allowed to return `null`, in order to signal

documentation/src/docs/asciidoc/user-guide/writing-tests.adoc

@@ -2347,6 +2347,8 @@ configuration parameter to one of the following options.
 `fixed`::
   Uses the mandatory `junit.jupiter.execution.parallel.config.fixed.parallelism`
   configuration parameter as the desired parallelism.
+  The optional `junit.jupiter.execution.parallel.config.fixed.max-pool-size`
+  configuration parameter can be used to limit the maximum number of threads.
 
 `custom`::
   Allows you to specify a custom `{ParallelExecutionConfigurationStrategy}`
@@ -2357,13 +2359,15 @@ If no configuration strategy is set, JUnit Jupiter uses the `dynamic` configurat
 strategy with a factor of `1`. Consequently, the desired parallelism will be equal to the
 number of available processors/cores.
 
-.Parallelism does not imply maximum number of concurrent threads
-NOTE: JUnit Jupiter does not guarantee that the number of concurrently executing tests
-will not exceed the configured parallelism. For example, when using one of the
-synchronization mechanisms described in the next section, the `ForkJoinPool` that is used
-behind the scenes may spawn additional threads to ensure execution continues with
-sufficient parallelism. Thus, if you require such guarantees in a test class, please use
-your own means of controlling concurrency.
+.Parallelism alone does not imply maximum number of concurrent threads
+NOTE: By default JUnit Jupiter does not guarantee that the number of concurrently
+executing tests will not exceed the configured parallelism. For example, when using one
+of the synchronization mechanisms described in the next section, the `ForkJoinPool` that
+is used behind the scenes may spawn additional threads to ensure execution continues with
+sufficient parallelism.
+If you require such guarantees, with Java 9+, it is possible to limit the maximum number
+of concurrent threads by controlling the maximum pool size of the `fixed` and `custom`
+strategies.
 
 [[writing-tests-parallel-execution-config-properties]]
 ===== Relevant properties
@@ -2415,6 +2419,20 @@ The following table lists relevant properties for configuring parallel execution
 | a positive integer
 | no default value
 
+| ```junit.jupiter.execution.parallel.config.fixed.max-pool-size```
+| Desired maximum pool size of the underlying fork-join pool for the ```fixed```
+  configuration strategy
+| a positive integer, must greater than or equal to `junit.jupiter.execution.parallel.config.fixed.parallelism`
+| 256 + the value of `junit.jupiter.execution.parallel.config.fixed.parallelism`
+
+| ```junit.jupiter.execution.parallel.config.fixed.saturate```
+| Disable saturation of the underlying fork-join pool for the ```fixed``` configuration
+  strategy
+|
+  * `true`
+  * `false`
+| ```true```
+
 | ```junit.jupiter.execution.parallel.config.custom.class```
 | Fully qualified class name of the _ParallelExecutionConfigurationStrategy_ to be
   used for the ```custom``` configuration strategy

junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/Constants.java

@@ -15,7 +15,9 @@
 import static org.apiguardian.api.API.Status.STABLE;
 import static org.junit.platform.engine.support.hierarchical.DefaultParallelExecutionConfigurationStrategy.CONFIG_CUSTOM_CLASS_PROPERTY_NAME;
 import static org.junit.platform.engine.support.hierarchical.DefaultParallelExecutionConfigurationStrategy.CONFIG_DYNAMIC_FACTOR_PROPERTY_NAME;
+import static org.junit.platform.engine.support.hierarchical.DefaultParallelExecutionConfigurationStrategy.CONFIG_FIXED_MAX_POOL_SIZE_PROPERTY_NAME;
 import static org.junit.platform.engine.support.hierarchical.DefaultParallelExecutionConfigurationStrategy.CONFIG_FIXED_PARALLELISM_PROPERTY_NAME;
+import static org.junit.platform.engine.support.hierarchical.DefaultParallelExecutionConfigurationStrategy.CONFIG_FIXED_SATURATE_PROPERTY_NAME;
 import static org.junit.platform.engine.support.hierarchical.DefaultParallelExecutionConfigurationStrategy.CONFIG_STRATEGY_PROPERTY_NAME;
 
 import org.apiguardian.api.API;
@@ -166,6 +168,40 @@ public final class Constants {
 	public static final String PARALLEL_CONFIG_FIXED_PARALLELISM_PROPERTY_NAME = PARALLEL_CONFIG_PREFIX
 			+ CONFIG_FIXED_PARALLELISM_PROPERTY_NAME;
 
+	/**
+	 * Property name used to configure the maximum pool size of the underlying
+	 * fork-join pool for the {@code fixed} configuration strategy: {@value}
+	 *
+	 * <p>Value must be an integer and greater than or equal to
+	 * {@value #PARALLEL_CONFIG_FIXED_PARALLELISM_PROPERTY_NAME}; defaults to
+	 * {@code 256 + fixed.parallelism}.
+	 *
+	 * <p>Note: This property only takes affect on Java 9+.
+	 *
+	 * @since 5.10
+	 */
+	@API(status = EXPERIMENTAL, since = "5.10")
+	public static final String PARALLEL_CONFIG_FIXED_MAX_POOL_SIZE_PROPERTY_NAME = PARALLEL_CONFIG_PREFIX
+			+ CONFIG_FIXED_MAX_POOL_SIZE_PROPERTY_NAME;
+
+	/**
+	 * Property name used to disable saturation of the underlying fork-join pool
+	 * for the {@code fixed} configuration strategy: {@value}
+	 *
+	 * <p>When set to {@code false} the underlying fork-join pool will reject
+	 * additional tasks if all available workers are busy and the maximum
+	 * pool-size would be exceeded.
+
+	 * <p>Value must either {@code true} or {@code false}; defaults to {@code true}.
+	 *
+	 * <p>Note: This property only takes affect on Java 9+.
+	 *
+	 * @since 5.10
+	 */
+	@API(status = EXPERIMENTAL, since = "5.10")
+	public static final String PARALLEL_CONFIG_FIXED_SATURATE_PROPERTY_NAME = PARALLEL_CONFIG_PREFIX
+			+ CONFIG_FIXED_SATURATE_PROPERTY_NAME;
+
 	/**
 	 * Property name used to set the factor to be multiplied with the number of
 	 * available processors/cores to determine the desired parallelism for the

junit-platform-engine/src/main/java/org/junit/platform/engine/support/hierarchical/DefaultParallelExecutionConfiguration.java

@@ -10,6 +10,9 @@
 
 package org.junit.platform.engine.support.hierarchical;
 
+import java.util.concurrent.ForkJoinPool;
+import java.util.function.Predicate;
+
 /**
  * @since 1.3
  */
@@ -20,14 +23,16 @@ class DefaultParallelExecutionConfiguration implements ParallelExecutionConfigur
 	private final int maxPoolSize;
 	private final int corePoolSize;
 	private final int keepAliveSeconds;
+	private final Predicate<? super ForkJoinPool> saturate;
 
 	DefaultParallelExecutionConfiguration(int parallelism, int minimumRunnable, int maxPoolSize, int corePoolSize,
-			int keepAliveSeconds) {
+			int keepAliveSeconds, Predicate<? super ForkJoinPool> saturate) {
 		this.parallelism = parallelism;
 		this.minimumRunnable = minimumRunnable;
 		this.maxPoolSize = maxPoolSize;
 		this.corePoolSize = corePoolSize;
 		this.keepAliveSeconds = keepAliveSeconds;
+		this.saturate = saturate;
 	}
 
 	@Override
@@ -55,4 +60,8 @@ public int getKeepAliveSeconds() {
 		return keepAliveSeconds;
 	}
 
+	@Override
+	public Predicate<? super ForkJoinPool> getSaturatePredicate() {
+		return saturate;
+	}
 }

junit-platform-engine/src/main/java/org/junit/platform/engine/support/hierarchical/DefaultParallelExecutionConfigurationStrategy.java

@@ -42,8 +42,14 @@ public ParallelExecutionConfiguration createConfiguration(ConfigurationParameter
 					() -> new JUnitException(String.format("Configuration parameter '%s' must be set",
 						CONFIG_FIXED_PARALLELISM_PROPERTY_NAME)));
 
-			return new DefaultParallelExecutionConfiguration(parallelism, parallelism, 256 + parallelism, parallelism,
-				KEEP_ALIVE_SECONDS);
+			int maxPoolSize = configurationParameters.get(CONFIG_FIXED_MAX_POOL_SIZE_PROPERTY_NAME,
+				Integer::valueOf).orElse(parallelism + 256);
+
+			boolean saturate = configurationParameters.get(CONFIG_FIXED_SATURATE_PROPERTY_NAME,
+				Boolean::valueOf).orElse(true);
+
+			return new DefaultParallelExecutionConfiguration(parallelism, parallelism, maxPoolSize, parallelism,
+				KEEP_ALIVE_SECONDS, __ -> saturate);
 		}
 	},
 
@@ -66,7 +72,7 @@ public ParallelExecutionConfiguration createConfiguration(ConfigurationParameter
 				factor.multiply(BigDecimal.valueOf(Runtime.getRuntime().availableProcessors())).intValue());
 
 			return new DefaultParallelExecutionConfiguration(parallelism, parallelism, 256 + parallelism, parallelism,
-				KEEP_ALIVE_SECONDS);
+				KEEP_ALIVE_SECONDS, null);
 		}
 	},
 
@@ -114,6 +120,36 @@ public ParallelExecutionConfiguration createConfiguration(ConfigurationParameter
 	 */
 	public static final String CONFIG_FIXED_PARALLELISM_PROPERTY_NAME = "fixed.parallelism";
 
+	/**
+	 * Property name used to configure the maximum pool size of the underlying
+	 * fork-join pool for the {@link #FIXED} configuration strategy.
+	 *
+	 * <p>Value must be an integer and greater than or equal to
+	 * {@value #CONFIG_FIXED_PARALLELISM_PROPERTY_NAME}; defaults to
+	 * {@code 256 + fixed.parallelism}.
+	 *
+	 * @since 1.10
+	 * @see #FIXED
+	 */
+	@API(status = EXPERIMENTAL, since = "1.10")
+	public static final String CONFIG_FIXED_MAX_POOL_SIZE_PROPERTY_NAME = "fixed.max-pool-size";
+
+	/**
+	 * Property name used to disable saturation of the underlying fork-join pool
+	 * for the {@link #FIXED} configuration strategy.
+	 *
+	 * <p>When set to {@code false} the underlying fork-join pool will reject
+	 * additional tasks if all available workers are busy and the maximum
+	 * pool-size would be exceeded.
+	 * <p>Value must either {@code true} or {@code false}; defaults to {@code true}.
+	 *
+	 * @since 1.10
+	 * @see #FIXED
+	 * @see #CONFIG_FIXED_MAX_POOL_SIZE_PROPERTY_NAME
+	 */
+	@API(status = EXPERIMENTAL, since = "1.10")
+	public static final String CONFIG_FIXED_SATURATE_PROPERTY_NAME = "fixed.saturate";
+
 	/**
 	 * Property name of the factor used to determine the desired parallelism for the
 	 * {@link #DYNAMIC} configuration strategy.

platform-tests/src/test/java/org/junit/platform/engine/support/hierarchical/DefaultParallelExecutionConfigurationStrategyTests.java

@@ -17,8 +17,6 @@
 import static org.mockito.Mockito.when;
 
 import java.util.Optional;
-import java.util.concurrent.ForkJoinPool;
-import java.util.function.Predicate;
 
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
@@ -51,7 +49,20 @@ void fixedStrategyCreatesValidConfiguration() {
 		assertThat(configuration.getMinimumRunnable()).isEqualTo(42);
 		assertThat(configuration.getMaxPoolSize()).isEqualTo(256 + 42);
 		assertThat(configuration.getKeepAliveSeconds()).isEqualTo(30);
-		assertThat(configuration.getSaturatePredicate()).isNull();
+		assertThat(configuration.getSaturatePredicate().test(null)).isTrue();
+	}
+
+	@Test
+	void fixedSaturateStrategyCreatesValidConfiguration() {
+		when(configParams.get("fixed.parallelism")).thenReturn(Optional.of("42"));
+		when(configParams.get("fixed.max-pool-size")).thenReturn(Optional.of("42"));
+		when(configParams.get("fixed.saturate")).thenReturn(Optional.of("false"));
+
+		ParallelExecutionConfigurationStrategy strategy = DefaultParallelExecutionConfigurationStrategy.FIXED;
+		var configuration = strategy.createConfiguration(configParams);
+		assertThat(configuration.getParallelism()).isEqualTo(42);
+		assertThat(configuration.getMaxPoolSize()).isEqualTo(42);
+		assertThat(configuration.getSaturatePredicate().test(null)).isFalse();
 	}
 
 	@Test
@@ -183,12 +194,7 @@ void customStrategyThrowsExceptionWhenClassDoesNotExist() {
 	static class CustomParallelExecutionConfigurationStrategy implements ParallelExecutionConfigurationStrategy {
 		@Override
 		public ParallelExecutionConfiguration createConfiguration(ConfigurationParameters configurationParameters) {
-			return new DefaultParallelExecutionConfiguration(1, 2, 3, 4, 5) {
-				@Override
-				public Predicate<? super ForkJoinPool> getSaturatePredicate() {
-					return __ -> true;
-				}
-			};
+			return new DefaultParallelExecutionConfiguration(1, 2, 3, 4, 5, __ -> true);
 		}
 	}