Skip to content

Fix only_new clear missing new tasks after a 'run on latest version' clear - #71687

Open
pulaksaha143 wants to merge 1 commit into
apache:mainfrom
pulaksaha143:fix-only-new-clear-71455
Open

Fix only_new clear missing new tasks after a 'run on latest version' clear#71687
pulaksaha143 wants to merge 1 commit into
apache:mainfrom
pulaksaha143:fix-only-new-clear-71455

Conversation

@pulaksaha143

Copy link
Copy Markdown

Resolves #71455.

Problem

When a user clears a DagRun with run_on_latest_version=True, the created_dag_version_id column is updated to the latest version. Later, if the user performs a clear with only_new=True, the _get_new_task_ids function diffs the latest DAG's task IDs against the task IDs from the version referenced by created_dag_version_id. Because this pointer has mutated, it incorrectly detects an empty diff and silently skips creating actual new tasks.

Solution

Instead of relying on the mutable created_dag_version_id, we now directly query the database for the actual TaskInstance rows associated with the DagRun. We diff these existing task IDs against the latest DAG's task IDs to accurately inject genuinely new tasks.

Also includes an automated test asserting correct behavior when created_dag_version_id has been updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

only_new clear can silently miss real new tasks after a prior "run on latest version" clear

1 participant