Checkpoint index reuse falls back on large sparse checkouts and still fails after #10792 #12080

Activity

1. juliusmarminge commented on Sep 16, 2026

   

   juliusmarminge

   on Sep 16, 2026

   Member

   More actions

   Triage: real server checkpoint bug. Confirmed on current main and on your 0.0.41-nightly.20260916.1795 (both include #10792). Shipping 0.0.42 does not include #10792 and is worse (always takes the fresh-index path). Not fixed by #10792, and not a wash of #3646.

   What you hit. On a large cone-mode sparse checkout, capture copies the index, runs read-tree --reset HEAD, then inspects git ls-files -v. Reuse only runs when that listing is under 16 MiB and has no assume-unchanged / skip-worktree flags (!entries.stdoutTruncated && !/^[a-zS] /m.test(...)). Your probe had ~40 MiB of listing and ~212k S entries — reuse rejected twice. Fallback read-tree HEAD left every entry H, then git add -A -- . hit the unchanged 30s deadline (31.1s). With a longer timeout the same add exits 1 because untracked files exist outside the sparse cone.

   Why #10792 does not save this. That PR intentionally falls back when S / lowercase flags remain or the listing is truncated — it was protecting manual update-index --skip-worktree / assume-unchanged. Cone-mode sparse-checkout is those S bits. So sparse monorepos never reuse, and the fallback expands the full tree.

   The add is also wrong, not just slow. git add -A -- . exits 1 on outside-cone untracked paths (reproduced on current Git for both workspace and fallback temp indexes). Raising the timeout alone would not make capture succeed while those artifacts exist. Fallback without reapplying skip-worktree is also semantically wrong: excluded paths become live H entries and can look like deletions. Open #11665 only retries transient exit failures (not timeouts) and would not fix this sparse-path exit.

   Related, not duplicates. #3646 is the umbrella large-repo 30s add -A timeout. #10792 is the partial reuse optimization that correctly opts out of this shape. #11665 is exit retries only. No open PR teaches capture about sparse-checkout.

   Suggested fix (in GitVcsDriver.checkpoints.captureCheckpoint). Detect sparse-checkout and keep / reuse skip-worktree (those S flags are the cone). Stop dumping a 40 MiB ls-files -v to decide reuse. Make git add sparse-safe (in-cone pathspecs or reapply sparse on the temp index; do not fail capture on outside-cone untracked warnings). If fallback still runs, reapply skip-worktree before add. Tests: cone-mode + sparse index + outside-cone untracked + in-cone edit; capture under 30s; workspace index unchanged; tree has the in-cone change and not 211k excluded deletions.

   Workaround. None in-app. Removing the six outside-cone files would only expose the 30s fallback timeout; reuse would still be rejected. Disabling sparse-checkout is not realistic here. Do not treat “update to 0.0.42”, “bump the timeout”, or “land #11665” as closing this.
2. juliusmarminge

   added

   bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.

   acceptedfeature request acceptedfeature request accepted

   via-triageFiled through npx t3 triageFiled through npx t3 triage

   on Sep 16, 2026
3. vedprakash2302 commented on Sep 16, 2026

   

   vedprakash2302

   on Sep 16, 2026

   ContributorAuthor

   More actions

   Opened #12154 to address this. It preserves sparse-checkout index reuse, removes the listing-size cutoff from the reuse decision, and handles files outside the sparse rules. Synthetic regression tests and benchmarks are in the PR.
4. juliusmarminge

   closed this as completedin #12154on Sep 17, 2026