[Feature Request]: Enhance Task Processing Efficiency

[fanyanpeng]

Is there an existing issue for the same feature request?

• I have checked the existing issues.

Is your feature request related to a problem?

Problem Statement:
I am frequently frustrated when using the system because it only processes one task at a time, which is particularly inefficient when dealing with a large volume of tasks.

Describe the feature you'd like

Currently, the system processes tasks one at a time, which is inefficient. I would like to request a feature that allows for batch processing of tasks to improve overall efficiency and speed.

Proposed Feature:
Concurrent Task Processing: Utilize multi-threading or multi-processing to handle tasks concurrently to increase processing speed.

Describe implementation you've considered

Enable the system to consume multiple tasks from the Redis queue at once, rather than processing tasks individually.

Documentation, adoption, use case

Additional information

No response