deferred / deferrable task are not take in account by max_active_tasks #40528
Labels
affected_version:2.9
area:async-operators
AIP-40: Deferrable ("Async") Operators
area:Triggerer
kind:bug
This is a clearly a bug
Apache Airflow version
2.9.2
What happened?
If you set max_active_tasks, it do not limit the number of concurrent deferred tasks.
It's possible to restrict the number of concurrent deferred tasks with a pool but I want limit the number of tasks run by a dag and keep the possibility for every task to use a specific pool ( we can only use 1 pool by task in airflow )
What you think should happen instead?
max_active_tasks should take in account the deferred task or we should create a new boolean setting
max_active_tasks_include_deferred
like what was done for the poolsHow to reproduce
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: