Rebase onto Git for Windows v2.53.0-rc2.windows.1#854
Merged
dscho merged 301 commits intomicrosoft:vfs-2.53.0-rc2from Jan 31, 2026
Merged
Rebase onto Git for Windows v2.53.0-rc2.windows.1#854dscho merged 301 commits intomicrosoft:vfs-2.53.0-rc2from
dscho merged 301 commits intomicrosoft:vfs-2.53.0-rc2from
Conversation
On index load, clear/set the skip worktree bits based on the virtual file system data. Use virtual file system data to update skip-worktree bit in unpack-trees. Use virtual file system data to exclude files and folders not explicitly requested. Update 2022-04-05: disable the "present-despite-SKIP_WORKTREE" file removal behavior when 'core.virtualfilesystem' is enabled. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Loosen the blocking of the `fsck` command from all "GVFS repos" (those that have `core.gvfs` set) to only those that actually use the virtual file system (VFS for Git only). This allows for `fsck` to be used in Scalar clones. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
…x has been redirected Fixes #13 Some git commands spawn helpers and redirect the index to a different location. These include "difftool -d" and the sequencer (i.e. `git rebase -i`, `git cherry-pick` and `git revert`) and others. In those instances we don't want to update their temporary index with our virtualization data. Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Loosen the blocking of the `prune` command from all "GVFS repos" (those that have `core.gvfs` set) to only those that actually use the virtual file system (VFS for Git only). This allows for `prune` to be used in Scalar clones. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Add check to see if a directory is included in the virtualfilesystem before checking the directory hashmap. This allows a directory entry like foo/ to find all untracked files in subdirectories.
Replace the special casing of the `worktree` command being blocked on VFS-enabled repos with the new `BLOCK_ON_VFS_ENABLED` flag. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
When our patches to support that hook were upstreamed, the hook's name was eliciting some reviewer suggestions, and it was renamed to `post-index-change`. These patches (with the new name) made it into v2.22.0. However, VFSforGit users may very well have checkouts with that hook installed under the original name. To support this, let's just introduce a hack where we look a bit more closely when we just failed to find the `post-index-change` hook, and allow any `post-indexchanged` hook to run instead (if it exists).
Emit a warning message when the `gvfs.sharedCache` option is set that the `repack` command will not perform repacking on the shared cache. In the future we can teach `repack` to operate on the shared cache, at which point we can drop this commit. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Teach STATUS to optionally serialize the results of a status computation to a file. Teach STATUS to optionally read an existing serialization file and simply print the results, rather than actually scanning. This is intended for immediate status results on extremely large repos and assumes the use of a service/daemon to maintain a fresh current status snapshot. 2021-10-30: packet_read() changed its prototype in ec9a37d (pkt-line.[ch]: remove unused packet_read_line_buf(), 2021-10-14). 2021-10-30: sscanf() now does an extra check that "%d" goes into an "int" and complains about "uint32_t". Replacing with "%u" fixes the compile-time error. 2021-10-30: string_list_init() was removed by abf897b (string-list.[ch]: remove string_list_init() compatibility function, 2021-09-28), so we need to initialize manually. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Teach status serialization to take an optional pathname on
the command line to direct that cache data be written there
rather than to stdout. When used this way, normal status
results will still be written to stdout.
When no path is given, only binary serialization data is
written to stdout.
Usage:
git status --serialize[=<path>]
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Teach status deserialize code to reject status cache when printing in porcelain V2 and there are unresolved conflicts in the cache file. A follow-on task might extend the cache format to include this additiona data. See code for longer explanation. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Changes to the global or repo-local excludes files can change the results returned by "git status" for untracked files. Therefore, it is important that the exclude-file values used during serialization are still current at the time of deserialization. Teach "git status --serialize" to report metadata on the user's global exclude file (which defaults to "$XDG_HOME/git/ignore") and for the repo-local excludes file (which is in ".git/info/excludes"). Serialize will record the pathnames and mtimes for these files in the serialization header (next to the mtime data for the .git/index file). Teach "git status --deserialize" to validate this new metadata. If either exclude file has changed since the serialization-cache-file was written, then deserialize will reject the cache file and force a full/normal status run. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Teach `git status --deserialize` to either wait indefintely or immediately fail if the status serialization cache file is stale. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
When using a virtual file system layer, the FSMonitor does not make sense. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With the "--untracked-files=complete" option status computes a superset of the untracked files. We use this when writing the status cache. If subsequent deserialize commands ask for either the complete set or one of the "no", "normal", or "all" subsets, it can still use the cache file because of filtering in the deserialize parser. When running status with the "-uno" option, the long format status would print a "(use -u to show untracked files)" hint. When deserializing with the "-uno" option and using a cache computed with "-ucomplete", the "nothing to commit, working tree clean" message would be printed instead of the hint. It was easy to miss because the correct hint message was printed if the cache was rejected for any reason (and status did the full fallback). The "struct wt_status des" structure was initialized with the content of the status cache (and thus defaulted to "complete"). This change sets "des.show_untracked_files" to the requested subset from the command-line or config. This allows the long format to print the hint. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
When sparse-checkout is enabled, add the sparse-checkout percentage to the Trace2 data stream. This number was already computed and printed on the console in the "You are in a sparse checkout..." message. It would be helpful to log it too for performance monitoring. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Add trace2 region around read_object_process to collect time spent waiting for missing objects to be dynamically fetched. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Add trace2 region and data events describing attempts to deserialize
status data using a status cache.
A category:status, label:deserialize region is pushed around the
deserialize code.
Deserialization results when reading from a file are:
category:status, path = <path>
category:status, polled = <number_of_attempts>
category:status, result = "ok" | "reject"
When reading from STDIN are:
category:status, path = "STDIN"
category:status, result = "ok" | "reject"
Status will fallback and run a normal status scan when a "reject"
is reported (unless "--deserialize-wait=fail").
If "ok" is reported, status was able to use the status cache and
avoid scanning the workdir.
Additionally, a cmd_mode is emitted for each step: collection,
deserialization, and serialization. For example, if deserialization
is attempted and fails and status falls back to actually computing
the status, a cmd_mode message containing "deserialize" is issued
and then a cmd_mode for "collect" is issued.
Also, if deserialization fails, a data message containing the
rejection reason is emitted.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Add trace information around status serialization. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
When using fsmonitor the CE_FSMONITOR_VALID flag should be checked when wanting to know if the entry has been updated. If the flag is set the entry should be considered up to date and the same as if the CE_UPTODATE is set. In order to trust the CE_FSMONITOR_VALID flag, the fsmonitor data needs to be refreshed when the fsmonitor bitmap is applied to the index in tweak_fsmonitor. Since the fsmonitor data is kept up to date for every command, some tests needed to be updated to take that into account. istate->untracked->use_fsmonitor was set in tweak_fsmonitor when the fsmonitor bitmap data was loaded and is now in refresh_fsmonitor since that is being called in tweak_fsmonitor. refresh_fsmonitor will only be called once and any other callers should be setting it when refreshing the fsmonitor data so that code can use the fsmonitor data when checking untracked files. When writing the index, fsmonitor_last_update is used to determine if the fsmonitor bitmap should be created and the extension data written to the index. When running through unpack-trees this is not copied to the result index. This makes the next time a git command is ran do all the work of lstating all files to determine what is clean since all entries in the index are marked as dirty since there wasn't any fsmonitor data saved in the index extension. Copying the fsmonitor_last_update to the result index will cause the extension data for fsmonitor to be in the index for the next git command to use. Signed-off-by: Kevin Willford <Kevin.Willford@microsoft.com>
Report virtual filesystem summary data. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
The fsmonitor script that can be used for running all the git tests using watchman was causing some of the tests to fail because it wrote to stderr and created some files for debugging purposes. Add a new debug script to use with debugging and modify the other script to remove the code that would cause tests to fail. Signed-off-by: Kevin Willford <Kevin.Willford@microsoft.com>
Create trace2_initialize_clock() and call from main() to capture process start time in isolation and before other sub-systems are ready. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Disable deserialization when verbose output requested. Verbose mode causes Git to print diffs for modified files. This requires the index to be loaded to have the currently staged OID values. Without loading the index, verbose output make it look like everything was deleted. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Verify that `git status --deserialize=x -v` does not crash and generates the same output as a normal (scanning) status command. These issues are described in the previous 2 commits. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Teach Git to not throw a fatal error when an explicitly-specified status-cache file (`git status --deserialize=<foo>`) could not be found or opened for reading and silently fallback to a traditional scan. This matches the behavior when the status-cache file is implicitly given via a config setting. Note: the current version causes a test to start failing. Mark this as an expected result for now. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Add trace2_thread_start() and trace2_thread_exit() events to the worker threads used to read the index. This gives per-thread perf data. These workers were introduced in: abb4bb8 read-cache: load cache extensions on a worker thread 77ff112 read-cache: load cache entries on worker threads Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
…or-windows#836) I'm exploring ideas around a new cache-server infrastructure. One of the trickiest parts of deploying new infrastructure is the need to have all endpoints ready to go at launch without incremental learning. This change adds new `gvfs.<verb>.cache-server` config keys that allow for verb-by-verb overrides of the typical `gvfs.cache-server` config key. These are loaded on a per-verb basis and then reset after the request. Further, if there is a failure then the request is retried with the base cache-server URL. This would allow us to, for example, deploy a service that serves only the `gvfs/prefetch` endpoint and see if that is improving on latency and throughput expectations before moving on to the GET and POST verbs for single and batched object downloads. As I was adding tests, I realized that we should split this test script into distinct parts so we can have a faster inner loop when testing specific areas. I know that this script is frequently the longest script running in our PR and CI builds, so the parallel split should help significantly. Use commit-by-commit review. I tried to keep the last two commits as obviously "copy-and-paste only" except for a small change to the port calculation to avoid overlap when using multiple ports in parallel tests. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
The `OI_DBCACHED` flag was removed in 7a4bd1b (packfile: always declare object info to be OI_PACKED, 2026-01-12). The commit message suggests to use the `is_delta` flag instead, but that is only populated if `OBJECT_INFO_QUICK` isn't passed (which is part of `OBJECT_INFO_FOR_PREFETCH`, which we use here). At the same time, no caller ever looked at the `cnt_dbcached` count. So let's just remove it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is needed because of 84f0e60 (packfile: move packfile store into object source, 2026-01-09). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The v2.53.0-rc0.vfs.0.0 build began failing with:
Undefined symbols for architecture arm64:
"_iconv", referenced from:
__libintl_find_msg in libintl.a[arm64][13](dcigettext.o)
"_iconv_open", referenced from:
__libintl_find_msg in libintl.a[arm64][13](dcigettext.o)
ld: symbol(s) not found for architecture arm64
See https://github.com/microsoft/git/actions/runs/21450597958 for the
failing run. The previous release v2.52.0.vfs.0.5 built successfully on
January 24, 2026 (https://github.com/microsoft/git/actions/runs/21313971421).
The root cause is upstream commit cee341e ("macOS: use iconv from
Homebrew if needed and present", 2025-12-24), which was included in Git
v2.53.0-rc0 to work around an iconv bug in macOS 15.7.2. That commit
introduced USE_HOMEBREW_LIBICONV, which config.mak.uname now sets on
Darwin 24+, causing the Makefile to add Homebrew's libiconv to the
linker search path.
The problem is a symbol name mismatch: Homebrew's libiconv exports
symbols with a prefix (_libiconv, _libiconv_open) to avoid conflicting
with the system library, but Homebrew's gettext/libintl was built
against the system iconv which uses unprefixed symbols (_iconv,
_iconv_open). When building universal binaries, the linker finds
Homebrew's libiconv first and cannot resolve the symbols libintl needs.
The fix is to explicitly unset USE_HOMEBREW_LIBICONV and ICONVDIR in
config.mak, forcing the linker to use the system's /usr/lib/libiconv.dylib
which is already universal and exports the correct unprefixed symbols.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Starting with upstream commit 4580bcd ("osxkeychain: avoid incorrectly skipping store operation", 2025-11-14), the osxkeychain credential helper includes git-compat-util.h and links against libgit.a. The osxkeychain Makefile has: CFLAGS ?= -g -O2 -Wall -I../../.. $(BASIC_CFLAGS) The ?= operator means "set only if not already set". When building via a parent Makefile that passes CFLAGS on the command line, this entire assignment is ignored - including the -I../../.. needed to find git-compat-util.h and the $(BASIC_CFLAGS) needed for -DNO_OPENSSL: git-credential-osxkeychain.c:5:10: fatal error: 'git-compat-util.h' file not found Using += instead of ?= is not sufficient either, because command-line variables in Make override even += assignments. We need to use "override CFLAGS +=" to force these flags to be appended regardless of how CFLAGS was set. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Starting with upstream commit 4580bcd ("osxkeychain: avoid incorrectly skipping store operation", 2025-11-14), the osxkeychain credential helper includes git-compat-util.h. That header includes openssl/ssl.h unless -DNO_OPENSSL is defined: ../../../git-compat-util.h:199:10: fatal error: 'openssl/ssl.h' file not found On macOS, the main Makefile sets NO_OPENSSL (because Apple Common Crypto is used instead) and adds -DNO_OPENSSL to BASIC_CFLAGS. But contrib Makefiles that only include config.mak* files don't get this logic - they only see the variables, not the Makefile rules that convert NO_OPENSSL into -DNO_OPENSSL. Add -DNO_OPENSSL to BASIC_CFLAGS in config.mak so that contrib builds like osxkeychain can pick it up. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
The previous fixup for 84f0e60 (packfile: move packfile store into object source, 2026-01-09) was incorrect. That upstream commit moved the packfile store from being a single global store in `struct object_database` to being per-source in `struct odb_source`. The previous fixup adapted the `packfile_store_reprepare()` call by changing: the_repository->objects->packfiles to: the_repository->objects->sources->packfiles However, this only reprepares the packfile store of the *first* source in the linked list. This is incorrect because GVFS-helper writes downloaded objects to `gh_client__chosen_odb`, which is typically the shared cache alternate (configured via `gvfs.sharedCache`), not the primary .git/objects directory. When objects are downloaded into the shared cache but only the primary source's packfile store is reprepared, the newly created packfiles remain invisible to subsequent object lookups. This causes the object lookup code in odb.c to fall through to the expensive `gh_client__get_immediate()` path, re-fetching objects that were already successfully downloaded via the batched queue mechanism. The test t5793.3 "integration: fully implicit: diff 2 commits" caught this regression. It verifies that blob objects are fetched via the efficient batched queue path (`gh_client__queue_oid`) and NOT via the single-object immediate path (`gh_client__get_immediate`). With the broken fixup, objects were being fetched twice: once via the queue (correct), and then again via immediate fetch (incorrect) because the packfile store for the shared cache was never reprepared. Fix this by repreparing the packfile store of the actual chosen ODB where objects are written: gh_client__chosen_odb->packfiles This variable is guaranteed to be set before `gh_client__objects__receive_response()` is called, since all callers go through `gh_client__find_long_running_process()` which calls `gh_client__choose_odb()` first. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This includes the fixes I had to make to get https://github.com/microsoft/git/releases/tag/v2.53.0-rc0.vfs.0.0 to build, plus a fix for an overlooked stale `GIT-VERSION-GEN`,
Member
Author
|
This should have been: Range-diff
|
This is now required due to `rs/tree-wo-the-repository`, specifically ec7a16b (cocci: convert parse_tree functions to repo_ variants, 2026-01-09). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
derrickstolee
approved these changes
Jan 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR rebases Microsoft Git patches onto Git for Windows v2.53.0-rc2.windows.1.
Previous base: vfs-2.53.0-rc1
Range-diff vs vfs-2.53.0-rc1
monitor-componentsworkflow in msft-gitgvfs.fallbackconfig settinguniversalconfigcommand for backwards compatibilityendpointcommandgvfs.sharedCachecache-servercommandclone --no-fetch-commits-and-treesfor backwards compatibilitygit stash -ucmdsizevariable is initializedlookup_commit()sane_istest()does not access array past endzinmallocz()strbuf_read()does NUL-terminate correctly-rc0