AIP-83 amendment: Add logic for generating run_id when logical date is None. #46616
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boring-cyborg
bot
added
area:API
sunank200
force-pushed
the
run_id_logic_with_no_logical_date
branch
3 times, most recently
from
6776668 to
25cfbf8
Compare
sunank200
added
the
legacy api
1 task
uranusjr
reviewed
Feb 10, 2025
uranusjr
reviewed
Feb 10, 2025
uranusjr
reviewed
Feb 10, 2025
uranusjr
reviewed
Feb 10, 2025
sunank200
force-pushed
the
run_id_logic_with_no_logical_date
branch
from
175ad1f to
fd91711
Compare
sunank200
force-pushed
the
run_id_logic_with_no_logical_date
branch
9 times, most recently
from
a351015 to
9b2116e
Compare
uranusjr
reviewed
Feb 11, 2025
uranusjr
reviewed
Feb 11, 2025
sunank200
force-pushed
the
run_id_logic_with_no_logical_date
branch
2 times, most recently
from
e6ccc9f to
2a4dc29
Compare
uranusjr
reviewed
Feb 11, 2025
uranusjr
reviewed
Feb 11, 2025
uranusjr
approved these changes
Feb 11, 2025
Member
uranusjr
left a comment
uranusjr
left a comment
There was a problem hiding this comment.
Two minor issues in test set up, otherwise this should be ready.
sunank200
force-pushed
the
run_id_logic_with_no_logical_date
branch
from
2a4dc29 to
5470414
Compare
Collaborator
Author
@uranusjr these should be fixed now. |
phanikumv
approved these changes
Feb 11, 2025
Contributor
|
I think we need to make run_after non public. Feels like we should not expose this to the user as an option -- but rather, it's something that the scheduler should figure out based on the dag / dag run. |
dstandish
reviewed
Feb 11, 2025
| dag_bag: DagBag, | ||
| *, | ||
| triggered_by: DagRunTriggeredByType, | ||
| run_after: datetime, |
Contributor
There was a problem hiding this comment.
so like e.g. this, not sure it makes sense to have this param
Contributor
There was a problem hiding this comment.
but basically, any public function or public endpoint, my thought is, probably we should not expose this to user as an option / param. essentially it's read only attr i'm thinking.
Collaborator
Author
There was a problem hiding this comment.
Created an issue here to fix this: #46650
ambika-garg
pushed a commit
to ambika-garg/airflow
that referenced
this pull request
Feb 17, 2025
…s None. (apache#46616) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
This was referenced Apr 6, 2025
NilsJPWerner
added a commit
to NilsJPWerner/airflow
that referenced
this pull request
Oct 3, 2025
In Airflow 3.x, commit 035060d (PR apache#46616) changed the trigger_dag module to use DagRun.generate_run_id() instead of dag.timetable.generate_run_id() for manual DAG runs. This bypassed custom timetable logic, causing a regression from Airflow 2.x behavior. This commit restores the use of dag.timetable.generate_run_id() for manual triggers, allowing custom timetables to control run_id generation for both scheduled and manually triggered runs. Changes: - airflow/api/common/trigger_dag.py: Changed to call dag.timetable.generate_run_id() with run_after and data_interval parameters instead of DagRun.generate_run_id() - tests/api_fastapi/core_api/routes/public/test_dag_run.py: Added test to verify custom timetable generate_run_id is called for manual triggers with both logical_date provided and null This pattern matches how manual triggers are handled in other parts of the codebase (e.g., assets.py, scheduler_job_runner.py). Fixes: apache#55908
NilsJPWerner
added a commit
to NilsJPWerner/airflow
that referenced
this pull request
Oct 3, 2025
In Airflow 3.x, commit 035060d (PR apache#46616) changed the trigger_dag module to use DagRun.generate_run_id() instead of dag.timetable.generate_run_id() for manual DAG runs. This bypassed custom timetable logic, causing a regression from Airflow 2.x behavior. This commit restores the use of dag.timetable.generate_run_id() for manual triggers, allowing custom timetables to control run_id generation for both scheduled and manually triggered runs. Changes: - airflow/api/common/trigger_dag.py: Changed to call dag.timetable.generate_run_id() with run_after and data_interval parameters instead of DagRun.generate_run_id() - tests/api_fastapi/core_api/routes/public/test_dag_run.py: Added test to verify custom timetable generate_run_id is called for manual triggers with both logical_date provided and null This pattern matches how manual triggers are handled in other parts of the codebase (e.g., assets.py, scheduler_job_runner.py). Fixes: apache#55908
pierrejeambrun
pushed a commit
to NilsJPWerner/airflow
that referenced
this pull request
Oct 14, 2025
In Airflow 3.x, commit 035060d (PR apache#46616) changed the trigger_dag module to use DagRun.generate_run_id() instead of dag.timetable.generate_run_id() for manual DAG runs. This bypassed custom timetable logic, causing a regression from Airflow 2.x behavior. This commit restores the use of dag.timetable.generate_run_id() for manual triggers, allowing custom timetables to control run_id generation for both scheduled and manually triggered runs. Changes: - airflow/api/common/trigger_dag.py: Changed to call dag.timetable.generate_run_id() with run_after and data_interval parameters instead of DagRun.generate_run_id() - tests/api_fastapi/core_api/routes/public/test_dag_run.py: Added test to verify custom timetable generate_run_id is called for manual triggers with both logical_date provided and null This pattern matches how manual triggers are handled in other parts of the codebase (e.g., assets.py, scheduler_job_runner.py). Fixes: apache#55908
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR started with changes on #46398. As the logic on 46398 has changed, I have created a different PR and closed the other PR
For the AIP 83 amendment, when a null logical date is provided, run_id is run_after + random string.
Closes: #46199
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.