Automatic Threads Configuration

Usecase

Multi-threaded job where the number of threads can be set or which sets the number automatically in relation to available processors.

Configuration Details

For general Maven, Spring and Database setup see the project setup wiki page.

There are 2 examples:

• full automatic configuration which checks number of available CPU to set number of threads
• half automatic configuration which works with system properties

Full Automatic Configuration

It is quite simple, just use a bean factory to create the taskexecutor with concurrency limit set in 1:1 relation to number of CPU

relevant sources:

<bean id="taskExecutor" 
  class="de.langmi.spring.batch.examples.autothreadconf.AsyncTaskExecutorFactory"
  factory-method="createInstance"/>

public class AsyncTaskExecutorFactory {
   ...
   public static SimpleAsyncTaskExecutor createInstance() {
       SimpleAsyncTaskExecutor instance = new SimpleAsyncTaskExecutor();

       // set concurrencyLimit according to available processors
       // real simple 1:1 relation
       Runtime runtime = Runtime.getRuntime();
       int nrCpu = runtime.availableProcessors();
       instance.setConcurrencyLimit(nrCpu);

       return instance;
   }
   ...
}

Half Automatic Configuration

This time the goold old fashioned PropertyPlaceholderConfigurer shines again.

relevant source:

<bean id="taskExecutor" class="de.langmi.spring.batch.examples.autothreadconf.LoggingAsyncTaskExecutor">
    <property name="concurrencyLimit" value="${concurrency.limit}" />
</bean>

<bean id="placeholderProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="location" value="classpath:spring/batch/job/autothreadconf/job.properties" />
    <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
    <property name="ignoreUnresolvablePlaceholders" value="false" />
</bean>

Why not late binding of Job Parameters ?

One could use late binding for the concurrency limit, but with late binding comes the step scope. In this simple example it is no problem, but imagine thread pooling and additional parallel steps, it would be a mess.