Skip to content

Conversation

@uranusjr
Copy link
Member

A DAG run with logical date in the future can never be started now. This only affects schedule=None, which can only be triggered manually.

Instead of using a future date, you can trigger with a None logical date whenever you want. A custom run_id can be supplied if you want it. If a date is needed, it can be passed as a DAG param instead.

@uranusjr uranusjr added the airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes label Feb 12, 2025
@boring-cyborg boring-cyborg bot added area:Scheduler including HA (high availability) scheduler area:task-sdk kind:documentation labels Feb 12, 2025
@Lee-W Lee-W self-requested a review February 12, 2025 04:25
A DAG run with logical date in the future can never be started now. This
only affects schedule=None, which can only be triggered manually.

Instead of using a future date, you can trigger with a None logical date
whenever you want. A custom run_id can be supplied if you want it. If a
date is needed, it can be passed as a DAG param instead.
@uranusjr uranusjr force-pushed the fix-logical-date-none-scheduler branch from 7512fb1 to 6f52ec2 Compare February 12, 2025 04:27
@uranusjr uranusjr force-pushed the fix-logical-date-none-scheduler branch from 6f52ec2 to 3ac9945 Compare February 12, 2025 04:30
@uranusjr uranusjr marked this pull request as ready for review February 12, 2025 06:28
@Lee-W
Copy link
Member

Lee-W commented Feb 12, 2025

I'll merge it and rebase it to #46460 once CI is green.

@uranusjr uranusjr merged commit 34261d0 into apache:main Feb 12, 2025
119 checks passed
@uranusjr uranusjr deleted the fix-logical-date-none-scheduler branch February 12, 2025 07:26
ambika-garg pushed a commit to ambika-garg/airflow that referenced this pull request Feb 17, 2025
A DAG run with logical date in the future can never be started now. This
only affects schedule=None, which can only be triggered manually.

Instead of using a future date, you can trigger with a None logical date
whenever you want. A custom run_id can be supplied if you want it. If a
date is needed, it can be passed as a DAG param instead.
@yuqian90
Copy link
Contributor

Since we have renamed execution_date to logical_date everywhere, does it still make sense to enforce that tasks with a future logical_date should not execute? logical_date should have no relationship to what time a task executes. It is just a date used to logically organize dagruns.

In my own use case, we organize dagruns using logical_date, and there are many template macros that rely on logical_date. Some tasks with logical_date 20250523 needs to execute before 20250523 00:00 UTC because they need to handle data for countries such as Australia or Japan. I also vaguely remember other use cases where Airflow is used to organize weather forecast data days in advance so using a future logical_date makes sense.

Regarding the suggestion "If a date is needed, it can be passed as a DAG param instead", this is not optimal because DAG param can't be a first class citizen in the UI. E.g. if we wanna filter by a date in the UI or sort dagruns by a date, we can't rely on a date that only resides in the param. Using logical_date for such purpose is exactly what "logical date" should be used for.

If there are no objections, I don't mind putting up a PR to remove this limitation of not allowing future logical_date to execute.

@ZhaoMJ
Copy link

ZhaoMJ commented Oct 16, 2025

We have similar usecases and I fully agree with @yuqian90.
I will add that the Calendar view is back in 3.1, but it won't work if we can't set a logical_date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes area:Scheduler including HA (high availability) scheduler area:task-sdk kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants