Jobs autoRun should have an max concurrent option

[matthijs166]

We’re using the Payload CMS Jobs system to queue tasks (e.g. AI generation). We have autoRun enabled with a 5-second interval to avoid rate-limit errors.

However, autoRun currently just triggers the next job every 5 seconds regardless of whether any jobs are still running.

Example of the problem:

• Our job limit is 5.
• A long-running job starts.
• 5 seconds later, autoRun kicks off another job even though the concurrency limit is already used.
• We risk overloading our external API, hitting rate limits, or running too many jobs simultaneously.

We’ve worked around this by creating a separate job whose only purpose is to check the active job count before starting new jobs, but this is awkward and feels like unnecessary complexity. Also, the logging in the terminal is very verbose.

We also checked shouldAutoRun, but it only passes the payload prop, so we can’t use it to check for active jobs, or we stop all runners from processing.

This is not a one-off problem, we keep encountering this pattern in multiple Payload projects where we want:

• A queue with a true concurrency limit enforced automatically.
• Jobs to start “as soon as possible,” but without exceeding the concurrency cap.

[matthijs166]

Looks like it's going to be supported with PR #12863

Exciting!!

[matthijs166]

Added in version: v3.49.0