-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
get_tree_view
can consume extreme amounts of memory.
#41505
Comments
This is a better reproduction DAG:
It uses 5+GB and takes just under 8 minutes to generate on my machine. And I believe it's less about the whitespace and more that we duplicate tasks in the output. For example, this DAG:
which results in this output:
Note how we get 9 |
It looks like |
I've marked the whole "tree" concept - the |
Apache Airflow version
2.10.0rc1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
get_tree_view
in degenerated case can take a lot of memory.For a DAG
serializing it can take 2.7GB
#41494
What you think should happen instead?
I think tree_view format should be changed to one that does not require extraordinary amount of whitespace in deeply nested cases.
Would be good to know in which cases it's being used though.
How to reproduce
You can use above dag.
Operating System
Docker/breeze on MacOS
Versions of Apache Airflow Providers
No response
Deployment
Other
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: