Fix hardcoded queue size in cloud executors

[ejseqera]

Fix for #6412 where hardcoded queue size values in cloud executors that were ignoring Nextflow configuration settings.

• Google Batch: Now uses config.getQueueSize(name, 1000) instead of hardcoded 1000
• Azure Batch: Now uses config.getQueueSize(name, 1000) instead of hardcoded 1000
• Kubernetes: Now uses config.getQueueSize(name, 100) instead of hardcoded 100

Previously, these executors ignored executor.queueSize or executor.$<executor>.queueSize settings, making config to throttle jobs ineffective.

This replaces hardcoded values with explicit configuration lookup while preserving original defaults that were set, similar to AWS Batch.

• Calls config.getQueueSize(name, defQueueSize)
• The defQueueSize parameter serves as the fallback default if no config is found/set
• Maintain configuration hierarchy: executor.$<executor>.queueSize → executor.queueSize → NXF_EXECUTOR_QUEUESIZE → defQueueSize

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	28bdc99
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/68cb929f8d5cdc0008699e14

[pditommaso]

Not sure what this improves because config.getQueueSize is invoked as first thing by the create method

nextflow/modules/nextflow/src/main/groovy/nextflow/processor/TaskPollingMonitor.groovy

Line 169 in 55f0913

final capacity = config.getQueueSize(name, defQueueSize)

[ejseqera]

@pditommaso Ah you're right, but then simplifying it to this should handle the config lookup issue correctly right?

TaskPollingMonitor.create(session, config, name, Duration.of('10 sec'))

[mikesivley]

In my attempt to resolve the throughput issue, I did have to update my nextflow version from 25.04.6. to the latest edge release so that I could modify and install the latest plugin from master. It's possible that this issue had already been resolved in master and that the nextflow update itself is what solved my issue, not my change to the plugin. I have not tested the behavior of 25.08.0-edge without the modified plugin.

Indeed, the lines within create to fetch the config values were added by your August 7th commit, but lacking from the last stable release. All is well in the edge release.

[mikesivley]

@ejseqera my original issue can be closed, but thank you for engaging with it so quickly!

[pditommaso]

Unless I didn't understood correctly the problem, this should not be needed

#6412 (comment)

[pditommaso]

Closing as not required. Please reopen or comment if the solution in #6412 does not solve for you