Skip to content

fetch-pack: parallelize no-ref-delta URI packs - #65

Merged
ttaylorr-oai merged 0 commit into
codexfrom
tb/codex/parallel-packfile-uris
Aug 24, 2026
Merged

fetch-pack: parallelize no-ref-delta URI packs#65
ttaylorr-oai merged 0 commit into
codexfrom
tb/codex/parallel-packfile-uris

Conversation

@ttaylorr-oai

@ttaylorr-oai ttaylorr-oai commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

This is the reduced stack for no-ref-delta URI-pack indexing.

  • Carries the four commits from
    tb/upstream/no-ref-delta
    first.
  • Keeps Friel's work as three commits: index-pack --no-ref-delta, v2
    fetch no-ref-delta support, and the parallel fetch path. Friel's author
    and Signed-off-by identity are preserved.
  • Uses one v2 fetch no-ref-delta contract for the inline pack and every
    URI pack. upload-pack passes --no-ref-delta to the inline
    pack-objects; the administrator must create or verify configured URI
    packs with the same property. The client requests the promise only with
    packfile URIs and fetch.packfileUriJobs > 1, verifies the inline pack
    and every URI pack with index-pack --no-ref-delta, and gives URI
    indexers --threads=1 before running them concurrently.
  • Uses the existing http-fetch --packfile helper. Expected keep files are
    created up front and entered in the existing pack-lock list before helpers
    start; pre-existing keeps are left alone. A small poll/refill loop bounds
    concurrent helpers. Direct download-to-index-pack streaming remains out
    of scope.

The final feature commit is 263 insertions / 11 deletions, down from
519 / 5 in the first reroll. The FIFO, background-job, polling-shell,
lying-server, duplicate-hash, one-job, invalid-config, and standalone
tempfile-cleanup tests were removed. The remaining coverage is
compositional: t5300 checks REF_DELTA rejection, and t5702 checks the
unified fetch negotiation, server-side pack-objects flag, inline verifier,
and three URI indexers with two jobs.

Testing

  • make -j8
  • (cd t && ./t5300-pack-object.sh --run=1-40) (selected tests pass)
  • (cd t && ./t5332-multi-pack-reuse.sh) (15/15)
  • (cd t && ./t5516-fetch-push.sh) (129/129)
  • (cd t && ./t5702-protocol-v2.sh) (88/88)

@ttaylorr-oai
ttaylorr-oai force-pushed the tb/codex/parallel-packfile-uris branch 3 times, most recently from 26ace25 to fb57f4a Compare August 21, 2026 00:09
@ttaylorr-oai
ttaylorr-oai force-pushed the tb/codex/parallel-packfile-uris branch 2 times, most recently from e077e6b to f9ed42f Compare August 21, 2026 16:49
@ttaylorr-oai
ttaylorr-oai marked this pull request as ready for review August 21, 2026 16:51
@ttaylorr-oai
ttaylorr-oai force-pushed the tb/codex/parallel-packfile-uris branch 2 times, most recently from e3fb229 to d94ac96 Compare August 21, 2026 19:56
@ttaylorr-oai
ttaylorr-oai force-pushed the tb/codex/parallel-packfile-uris branch from d94ac96 to a08b8d8 Compare August 22, 2026 19:19
@ttaylorr-oai
ttaylorr-oai merged commit b1e3ec2 into codex Aug 24, 2026
115 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants