EdgeModifier refactoring #21404
EdgeModifier refactoring #21404
Conversation
avkirilishin
force-pushed
the
label_refactoring_issue_17469
branch
2 times, most recently
from
189b221 to
4e810c3
Compare
|
I think this one needs a deeper look @bbovenzi @kaxil - I think you asked for a fix in #17469 and it LOOKS like a good fix. Which (if confirmed) might even qualify for 2.2.4 release @jedcunningham ? (marked it provisionally). I do not know that well about EdgeModifiers and relations setting but it does look legit from first look. And I think possibly some unit tests here woudl be useful. WDYT? |
|
@avkirilishin - would it be possible to add some unit tests for this one ? |
I am ready to add some unit tests if the approach is OK. |
|
Yeah, I would say we should move forward with this approach. |
|
Also good for me so @avkirilishin -> adding some testswill be great. |
@potiuk Done |
github-actions
bot
added
the
full tests needed
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
avkirilishin
force-pushed
the
label_refactoring_issue_17469
branch
from
4239c97 to
a7ef3ba
Compare
potiuk
force-pushed
the
label_refactoring_issue_17469
branch
from
ccb1bf1 to
a023dc9
Compare
|
Rebasing to see if issues are fixed (should be by MS). |
|
Jsut a temporary error that needs to be handled separately. |
This reverts commit ace8c6e. Fixes apache#23285 This dag breaks with a cycle as group.end has itself as a downstream otherwise: ```python from pendulum import datetime from airflow.decorators import dag, task, task_group from airflow.utils.edgemodifier import Label @task def begin(): ... @task def end(): ... @dag(start_date=datetime(2022, 1, 1), schedule_interval=None) def task_groups_with_edge_labels(): @task_group def group(): begin() >> end() group() _ = task_groups_with_edge_labels() ```
This reverts commit ace8c6e. Fixes #23285 This dag breaks with a cycle as group.end has itself as a downstream otherwise: ```python from pendulum import datetime from airflow.decorators import dag, task, task_group from airflow.utils.edgemodifier import Label @task def begin(): ... @task def end(): ... @dag(start_date=datetime(2022, 1, 1), schedule_interval=None) def task_groups_with_edge_labels(): @task_group def group(): begin() >> end() group() _ = task_groups_with_edge_labels() ```
This reverts commit ace8c6e. Fixes #23285 This dag breaks with a cycle as group.end has itself as a downstream otherwise: ```python from pendulum import datetime from airflow.decorators import dag, task, task_group from airflow.utils.edgemodifier import Label @task def begin(): ... @task def end(): ... @dag(start_date=datetime(2022, 1, 1), schedule_interval=None) def task_groups_with_edge_labels(): @task_group def group(): begin() >> end() group() _ = task_groups_with_edge_labels() ``` (cherry picked from commit 3182303)
This reverts commit ace8c6e. Fixes #23285 This dag breaks with a cycle as group.end has itself as a downstream otherwise: ```python from pendulum import datetime from airflow.decorators import dag, task, task_group from airflow.utils.edgemodifier import Label @task def begin(): ... @task def end(): ... @dag(start_date=datetime(2022, 1, 1), schedule_interval=None) def task_groups_with_edge_labels(): @task_group def group(): begin() >> end() group() _ = task_groups_with_edge_labels() ``` (cherry picked from commit 3182303)
6 participants
closes: #17469
Tested on dags:
test_label_refactoring.py.zip
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.