Skip to content

Why is Local Chunking in spring-batch-integration instead of spring-batch-core? #5185

@KMGeon

Description

@KMGeon

Hello, I've been studying the new Local Chunking feature added in Spring Batch 6.0, and I'm curious about why this feature is located in spring-batch-integration rather than spring-batch-core.

Definition of Integration

According to the [Spring Batch Integration documentation]

"Many users of Spring Batch may encounter requirements that are outside the scope of Spring Batch but that may be efficiently and concisely implemented by using Spring Integration."

In other words, the spring-batch-integration module is intended to implement requirements that are outside the scope of Spring Batch by leveraging Spring Integration (external messaging systems).

The key features in this module include:

  • Remote Chunking: Delegating chunk processing to remote workers via message queues (RabbitMQ, Kafka, etc.)
  • Remote Partitioning: Executing partitions on external nodes
  • Spring Integration integration: Message-based Job execution, event handling, etc.

All of these deal with integration with external systems (message brokers, remote nodes).

Characteristics of Local Chunking

Local Chunking, introduced in [#5021]

"The idea is to introduce an item writer similar to ChunkMessageChannelItemWriter, but instead of sending chunk requests over the wire to remote workers, it would submit them to local worker threads from a task executor."

Key characteristics:

  • Operates within a single JVM
  • Uses TaskExecutor (thread pool) to delegate chunks to local worker threads
  • No dependency on external systems (message brokers, remote nodes)
  • TaskExecutor is a basic Spring Framework component, unrelated to Spring Integration

Question

Local Chunking:

  • Does not require integration with external systems
  • Has no dependency on Spring Integration
  • Only uses TaskExecutor, which is already used in spring-batch-core for Multi-threaded Step, etc.

So why is this feature located in spring-batch-integration rather than spring-batch-core?

The item-level concurrency introduced in [#4955](ChunkOrientedStep's processChunkConcurrently) is in spring-batch-core, but Local Chunking (chunk-level concurrency), which similarly only uses TaskExecutor, is in spring-batch-integration. This seems inconsistent.

Understanding the background of this design decision would be very helpful for understanding Spring Batch.

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions