Merge pull request #410: Sparse Index: latest integrations

```
6e74958 p2000: add 'git checkout -' test and decrease depth
3e1d03c p2000: compress repo names
cd94f82 commit: integrate with sparse-index
65e79b8 sparse-index: recompute cache-tree
e9a9981 checkout: stop expanding sparse indexes
4b801c8 t1092: document bad 'git checkout' behavior
71e3015 unpack-trees: resolve sparse-directory/file conflicts
5e96df4 t1092: test merge conflicts outside cone
defab1b add: allow operating on a sparse-only index
9fc4313 pathspec: stop calling ensure_full_index
0ec03ab add: ignore outside the sparse-checkout in refresh()
adf5b15 add: remove ensure_full_index() with --renormalize
```

These commits are equivalent to those already in `next` via gitgitgadget#999.

```
80b8d6c Merge branch 'sparse-index/add' into stolee/sparse-index/add
```

This merge resolves conflicts with some work that happened in parallel, but is already in upstream `master`.

```
c407b2c t7519: rewrite sparse index test
9dad0d2 sparse-index: silently return when not using cone-mode patterns
2974920 sparse-index: silently return when cache tree fails
e7cdaa0 unpack-trees: fix nested sparse-dir search
347410c sparse-checkout: create helper methods
4537233 attr: be careful about sparse directories
5282a86 sparse-index: add SPARSE_INDEX_MEMORY_ONLY flag
3a2f316 sparse-checkout: clear tracked sparse dirs
fb47b56 sparse-checkout: add config to disable deleting dirs
```

These commits are the ones under review as of gitgitgadget#1009. Recent review made this less stable. It's a slightly different and more robust version of #396.

> Note: I'm still not done with the feedback for upstream, but the remaining feedback is "can we add tests that cover these tricky technical bits?" and in `microsoft/git` these are already covered by the Scalar functional tests (since that's how they were found).

```
080b02c diff: ignore sparse paths in diffstat
d91a647 merge: make sparse-aware with ORT
df49b5f merge-ort: expand only for out-of-cone conflicts
cdecb85 t1092: add cherry-pick, rebase tests
0c1ecfb sequencer: ensure full index if not ORT strategy
406dfbe sparse-index: integrate with cherry-pick and rebase
```

These commits integrate with `git merge`, `git cherry-pick`, `git revert`, and `git rebase` as of gitgitgadget#1019. This got some feedback that changed how the tests were working so they are more robust. This led to a new commit (0c1ecfb).

```
cbb0ab3 Merge branch 'sparse-index/merge' into vfs-2.33.0
acb8623 t7524: test no longer fails
```

Finally, the commits are merged into `vfs-2.33.0` and also we include a fix to a `microsoft/git` test that is no longer broken.

Cc: @vdye and @ldennington to get a (possibly overwhelming?) taste of sparse-index stuff. If you focus solely on the `git merge` commits you'll get a feel for what a sparse index integration looks like.

Author: derrickstolee

Documentation/config/index.txt

@@ -1,3 +1,9 @@
+index.deleteSparseDirectories::
+	When enabled, the cone mode sparse-checkout feature will delete
+	directories that are outside of the sparse-checkout cone, unless
+	such a directory contains an untracked, non-ignored file. Defaults
+	to true.
+
 index.recordEndOfIndexEntries::
 	Specifies whether the index file should include an "End Of Index
 	Entry" section. This reduces index load time on multiprocessor

builtin/sparse-checkout.c

@@ -102,7 +102,7 @@ static int sparse_checkout_list(int argc, const char **argv)
 
 static void clean_tracked_sparse_directories(struct repository *r)
 {
-	int i, was_full = 0;
+	int i, value, was_full = 0;
 	struct strbuf path = STRBUF_INIT;
 	size_t pathlen;
 	struct string_list_item *item;
@@ -118,6 +118,13 @@ static void clean_tracked_sparse_directories(struct repository *r)
 	    !r->index->sparse_checkout_patterns->use_cone_patterns)
 		return;
 
+	/*
+	 * Users can disable this behavior.
+	 */
+	if (!repo_config_get_bool(r, "index.deletesparsedirectories", &value) &&
+	    !value)
+		return;
+
 	/*
 	 * Use the sparse index as a data structure to assist finding
 	 * directories that are safe to delete. This conversion to a

diff.c

@@ -3901,6 +3901,13 @@ static int reuse_worktree_file(struct index_state *istate,
 	if (!FAST_WORKING_DIRECTORY && !want_file && has_object_pack(oid))
 		return 0;
 
+	/*
+	 * If this path does not match our sparse-checkout definition,
+	 * then the file will not be in the working directory.
+	 */
+	if (!path_in_sparse_checkout(name, istate))
+		return 0;
+
 	/*
 	 * Similarly, if we'd have to convert the file contents anyway, that
 	 * makes the optimization not worthwhile.

t/t1091-sparse-checkout-builtin.sh

@@ -673,6 +673,10 @@ test_expect_success 'cone mode clears ignored subdirectories' '
 	git -C repo status --porcelain=v2 >out &&
 	test_must_be_empty out &&
 
+	git -C repo -c index.deleteSparseDirectories=false sparse-checkout reapply &&
+	test_path_is_dir repo/folder1 &&
+	test_path_is_dir repo/deep/deeper2 &&
+
 	git -C repo sparse-checkout reapply &&
 	test_path_is_missing repo/folder1 &&
 	test_path_is_missing repo/deep/deeper2 &&

t/t1092-sparse-checkout-compatibility.sh

@@ -766,6 +766,23 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '
 	)
 '
 
+test_expect_success 'sparse-index is not expanded: merge conflict in cone' '
+	init_repos &&
+
+	for side in right left
+	do
+		git -C sparse-index checkout -b expand-$side base &&
+		echo $side >sparse-index/deep/a &&
+		git -C sparse-index commit -a -m "$side" || return 1
+	done &&
+
+	(
+		sane_unset GIT_TEST_MERGE_ALGORITHM &&
+		git -C sparse-index config pull.twohead ort &&
+		ensure_not_expanded ! merge -m merged expand-right
+	)
+'
+
 # NEEDSWORK: a sparse-checkout behaves differently from a full checkout
 # in this scenario, but it shouldn't.
 test_expect_success 'reset mixed and checkout orphan' '

t/t7524-serialized-status.sh

@@ -400,7 +400,7 @@ EOF
 
 '
 
-test_expect_failure 'ensure deserialize -v does not crash' '
+test_expect_success 'ensure deserialize -v does not crash' '
 
 	git init -b main verbose_test &&
 	touch verbose_test/a &&