Empty Graph view for Consecutive Dynamic Task Group Mapping #29287
Description
Apache Airflow version
2.5.1
What happened
The graph view disappear when launching the dag in the UI. It will start running, but fail to show the graph. In the Tree view, I can see the first mapped task_group executing, but the second one doesn't run. In the DAG Runs listing, the dag reports as failed.
What you think should happen instead
I would expect the graph view to keep showing, and the second task_group run when the last task of the first group finishes.
How to reproduce
from airflow.decorators import task, task_group
from airflow import DAG
import pendulum
from string import ascii_lowercase
from itertools import repeat
HUGE_KWARG = {f"kw-{x}": ascii_lowercase[x] * 1024 for x in range(10)}
@task
def sample_run(value="foo-bar"):
return value
@task_group
def task_group_map(huge_kwarg):
t1 = sample_run(value=huge_kwarg)
t2 = sample_run(t1)
t3 = sample_run(t2)
t4 = sample_run(t3)
t5 = sample_run(t4)
t6 = sample_run(t5)
t7 = sample_run(t6)
t8 = sample_run(t7)
t9 = sample_run(t8)
return sample_run(t9)
with DAG(
dag_id=f"task_group_mapping",
start_date=pendulum.datetime(2023, 1, 1, 0, tz='UTC'),
end_date=pendulum.datetime(2023, 1, 1, 6, tz='UTC'),
schedule="*/10 * * * *",
catchup=True,
tags=["sandbox"],
max_active_runs=1,
) as dag:
huge_kwarg = task_group_map.expand(huge_kwarg=list(repeat(HUGE_KWARG, 2)))
huge_kwarg = task_group_map.expand(huge_kwarg=huge_kwarg)Operating System
Ubuntu 20.04.5 LTS
Versions of Apache Airflow Providers
❯ pip freeze | grep apache-airflow
apache-airflow==2.5.1
apache-airflow-providers-common-sql==1.3.3
apache-airflow-providers-ftp==3.3.0
apache-airflow-providers-http==4.1.1
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-slack==7.2.0
apache-airflow-providers-sqlite==3.3.1
Deployment
Docker-Compose
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct