Skip to content

Fix GitDagBundle re-cloning on every task when prune_dotgit_folder is True#61847

Merged
ephraimbuddy merged 2 commits intoapache:mainfrom
astronomer:fix-prune-dotgit-folder
Feb 16, 2026
Merged

Fix GitDagBundle re-cloning on every task when prune_dotgit_folder is True#61847
ephraimbuddy merged 2 commits intoapache:mainfrom
astronomer:fix-prune-dotgit-folder

Conversation

@ephraimbuddy
Copy link
Contributor

@ephraimbuddy ephraimbuddy commented Feb 13, 2026

When a version directory already existed without a .git folder, workers treated it as invalid and deleted it then re-cloned. Tasks using the same bundle version would trigger a full clone on every run.

Detect an existing pruned worktree and reuse it instead of cloning again

closes: #61396

@ephraimbuddy ephraimbuddy changed the title Fix GitDagBundle re-cloning on every task when prune_dotgit_folder is… Fix GitDagBundle re-cloning on every task when prune_dotgit_folder is True Feb 13, 2026
… True

When a version directory already existed without a .git folder, workers treated
it as invalid and deleted it then re-cloned. Tasks using the same bundle version
would trigger a full clone on every run.

Detect an existing pruned worktree and reuse it instead of cloning again
@ephraimbuddy ephraimbuddy merged commit 3fe68e7 into apache:main Feb 16, 2026
247 of 248 checks passed
@ephraimbuddy ephraimbuddy deleted the fix-prune-dotgit-folder branch February 16, 2026 07:16
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
… True (apache#61847)

* Fix GitDagBundle re-cloning on every task when prune_dotgit_folder is True

When a version directory already existed without a .git folder, workers treated
it as invalid and deleted it then re-cloned. Tasks using the same bundle version
would trigger a full clone on every run.

Detect an existing pruned worktree and reuse it instead of cloning again

* fixup! Fix GitDagBundle re-cloning on every task when prune_dotgit_folder is True
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Constant Inode flips on LocalExecutor cause task failures (FileNotFoundError)

2 participants