fetch-pack: parallelize no-ref-delta URI packs - #65
Merged
Conversation
ttaylorr-oai
force-pushed
the
tb/codex/parallel-packfile-uris
branch
3 times, most recently
from
August 21, 2026 00:09
26ace25 to
fb57f4a
Compare
ttaylorr-oai
force-pushed
the
tb/codex/parallel-packfile-uris
branch
2 times, most recently
from
August 21, 2026 16:49
e077e6b to
f9ed42f
Compare
ttaylorr-oai
marked this pull request as ready for review
August 21, 2026 16:51
ttaylorr-oai
force-pushed
the
tb/codex/parallel-packfile-uris
branch
2 times, most recently
from
August 21, 2026 19:56
e3fb229 to
d94ac96
Compare
friel-openai
approved these changes
Aug 21, 2026
ttaylorr-oai
force-pushed
the
tb/codex/parallel-packfile-uris
branch
from
August 22, 2026 19:19
d94ac96 to
a08b8d8
Compare
friel-openai
approved these changes
Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the reduced stack for no-ref-delta URI-pack indexing.
tb/upstream/no-ref-delta
first.
index-pack --no-ref-delta, v2fetch
no-ref-deltasupport, and the parallel fetch path. Friel's authorand
Signed-off-byidentity are preserved.no-ref-deltacontract for the inline pack and everyURI pack.
upload-packpasses--no-ref-deltato the inlinepack-objects; the administrator must create or verify configured URIpacks with the same property. The client requests the promise only with
packfile URIs and
fetch.packfileUriJobs > 1, verifies the inline packand every URI pack with
index-pack --no-ref-delta, and gives URIindexers
--threads=1before running them concurrently.http-fetch --packfilehelper. Expected keep files arecreated up front and entered in the existing pack-lock list before helpers
start; pre-existing keeps are left alone. A small poll/refill loop bounds
concurrent helpers. Direct download-to-
index-packstreaming remains outof scope.
The final feature commit is 263 insertions / 11 deletions, down from
519 / 5 in the first reroll. The FIFO, background-job, polling-shell,
lying-server, duplicate-hash, one-job, invalid-config, and standalone
tempfile-cleanup tests were removed. The remaining coverage is
compositional:
t5300checks REF_DELTA rejection, andt5702checks theunified fetch negotiation, server-side
pack-objectsflag, inline verifier,and three URI indexers with two jobs.
Testing
make -j8(cd t && ./t5300-pack-object.sh --run=1-40)(selected tests pass)(cd t && ./t5332-multi-pack-reuse.sh)(15/15)(cd t && ./t5516-fetch-push.sh)(129/129)(cd t && ./t5702-protocol-v2.sh)(88/88)