Dynamic Task Mapping Skips Before Upstream Has Started #59289
Description
Apache Airflow version
Other Airflow 2/3 version (please specify below)
If "Other Airflow 2/3 version" selected, which one?
3.0.6
What happened?
"Dynamic Task Mapping Skips Before Upstream Has Started"
This is a similar bug as #28973
Difference is that adding "trigger_rule": TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS makes the bug happens.
What you think should happen instead?
Should not skip tasks
How to reproduce
from datetime import datetime
from airflow import DAG
from airflow.operators.empty import EmptyOperator
from airflow.decorators import dag, task, task_group
from airflow.utils.trigger_rule import TriggerRule
@task
def get_the_list():
return ["one", "two", "three"]
@task
def process_something(x: str, y: str, **kwargs):
output = f"{x}_{y}"
print(output)
return output
@task_group
def the_task_group(x: str, y: str, **kwargs):
start = EmptyOperator(task_id="start")
something = process_something(x, y)
another = process_something(something)
end = EmptyOperator(task_id="end")
start >> something >> another >> end
@dag(
default_args={
"start_date": datetime(year=2025, month=12, day=1),
"trigger_rule": TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS,
}
)
def dtm_bug_reproduce_dag():
start = EmptyOperator(task_id="start")
end = EmptyOperator(task_id="end")
the_list = get_the_list()
mapped_group = (
the_task_group
.partial(x="base")
.expand(y=the_list)
)
start >> the_list >> mapped_group >> end
dag = dtm_bug_reproduce_dag()
Operating System
MWAA
Versions of Apache Airflow Providers
No response
Deployment
Amazon (AWS) MWAA
Deployment details
No response
Anything else?
I was able to reproduce the bug with the attached snippet in 3.0.6. I haven't tried it in the latest version but I did search through the issues and haven't found it reported so I think it's likely that it's not yet fixed.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct