-
Notifications
You must be signed in to change notification settings - Fork 15.5k
Description
Description
When using decorators to define dags, e.g. dag_1.py:
from airflow.decorators import dag, task
from airflow.utils.dates import days_ago
DEFAULT_ARGS = {
"owner": "airflow",
}
@task
def some_task():
pass
@dag(
default_args=DEFAULT_ARGS,
schedule_interval=None,
start_date=days_ago(2),
)
def my_dag():
some_task()
DAG_1 = my_dag()
and
dag_2.py:
from airflow.decorators import dag, task
from airflow.utils.dates import days_ago
DEFAULT_ARGS = {
"owner": "airflow",
}
@task
def some_other_task():
pass
@dag(
default_args=DEFAULT_ARGS,
schedule_interval=None,
start_date=days_ago(2),
)
def my_dag():
some_other_task()
DAG_2 = my_dag()
We have two different dags which have been written in isolation, but by sheer bad luck both define my_dag()
. This seems fine for each file in isolation, but on the airflow UI, we only end up seeing one entry for my_dag
, where it has picked up dag_1.py
and ignored dag_2.py
.
Use case / motivation
We currently end up with only one DAG showing up on the UI, and no indication as to why the other one hasn't appeared.
Suggestion: popup similar to 'DAG import error' to highlight what needs changing in one of the DAG files in order for both to show up ("DAG import error: duplicate dag names found - please review {duplicate files} and ensure all dag definitions are unique"?)
Are you willing to submit a PR?
No time to spare on this at present
Related Issues
I haven't found any related issues with the search function.