-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
Hello, I've been studying the new Local Chunking feature added in Spring Batch 6.0, and I'm curious about why this feature is located in spring-batch-integration rather than spring-batch-core.
Definition of Integration
According to the [Spring Batch Integration documentation]
"Many users of Spring Batch may encounter requirements that are outside the scope of Spring Batch but that may be efficiently and concisely implemented by using Spring Integration."
In other words, the spring-batch-integration module is intended to implement requirements that are outside the scope of Spring Batch by leveraging Spring Integration (external messaging systems).
The key features in this module include:
- Remote Chunking: Delegating chunk processing to remote workers via message queues (RabbitMQ, Kafka, etc.)
- Remote Partitioning: Executing partitions on external nodes
- Spring Integration integration: Message-based Job execution, event handling, etc.
All of these deal with integration with external systems (message brokers, remote nodes).
Characteristics of Local Chunking
Local Chunking, introduced in [#5021]
"The idea is to introduce an item writer similar to ChunkMessageChannelItemWriter, but instead of sending chunk requests over the wire to remote workers, it would submit them to local worker threads from a task executor."
Key characteristics:
- Operates within a single JVM
- Uses TaskExecutor (thread pool) to delegate chunks to local worker threads
- No dependency on external systems (message brokers, remote nodes)
- TaskExecutor is a basic Spring Framework component, unrelated to Spring Integration
Question
Local Chunking:
- Does not require integration with external systems
- Has no dependency on Spring Integration
- Only uses TaskExecutor, which is already used in spring-batch-core for Multi-threaded Step, etc.
So why is this feature located in spring-batch-integration rather than spring-batch-core?
The item-level concurrency introduced in [#4955](ChunkOrientedStep's processChunkConcurrently) is in spring-batch-core, but Local Chunking (chunk-level concurrency), which similarly only uses TaskExecutor, is in spring-batch-integration. This seems inconsistent.
Understanding the background of this design decision would be very helpful for understanding Spring Batch.
Thank you!