Create gitexport CLI

Summary:
Creating an initial version of the gitexport CLI (for more context on why we need it, see T160586594).

This tool is supposed to take a repo and a list of paths as input and it should export all the history of those paths in a git repo.

## What does it do now?
Currently, this binary doesn't do anything useful. it just gets the history of a single path to be exported and prints their changeset ids and commit messages (for manual debugging).

The main point of this diff is to **set most of the structure/flow of the tool to get some early feedback** before I start implementing anything more complex.

Most of the functions don't have an actual implementation, but just do something simple (e.g. returning the first element of a vector) so it typechecks.

## What's my current plan?
1) Get the history of all the given paths. (This is mostly done in this diff already)
2) Merge the changesets into a single, topologically sorted, list of changesets
3) Strip irrelevant changes from every commit (T161205476).
4) Create a CommitGraph from this list (T161204758).
5) Export that CommitGraph to a new, temporary, Mononoke repo (T160787114).
6) Use existing tools to export that temporary repo to a git repo (T160787114).

The tricky bits are steps 2,3 and 4, which is where I expect to spend most of my time.

First, I'm not sure if event to create a CommitGraph at all, to be able to export the processed changesets to a new repo. If I do need to, I'm not sure if I should (a) strip the irrelevant file changes before or after creating the graph and (b) how to create a new repo and populate it with the commits from the graph I created. (b) is more of a implementation detail, so I don't worry about now...

The main unknowns for me are #2 and #4. Basically, how can I create a proper commit graph from a set of commits that are not direct descendants of each other. Assuming a linear history, I don't think it would be very complicated, but we also have to support branching, so I'm not sure how to do this efficiently...

## Examples
Let me put as simple example below. Commits with uppercase letters are relevant (i.e. should be exported) and lower case letters should now.

```
A -> b -> C -> D -> e
|-> f -> G
```

In this case, I want to have the following commit graph in the end:
```
A' -> C' -> D'
|-> G'
```
where X' is X stripped of irrelevant changes

## RFC
- This is my first Rust diff ever, so please LMK what horrible things I'm doing, bc I'm very likely doing a few 😂
- Does the plan I described above make sense?
- Any suggestions/ideas on how to efficiently stitch the changesets together would be appreciated! But I'll probably set up some time to discuss this problem specifically once I spend more time thinking about it...

## Next steps
- Implement steps #5 and #6 (T160787114) to get the entire E2E solution working with the simplest case (i.e. one path with linear history). This is basically exporting the commit graph to a git repo (maybe through a temporary mononoke repo).
  - Update integration test case to actually run and test the tool with the simple case.
- Figure out how to properly create a commit graph from a list of changeset lists.
- Add test cases for multiple paths and edge cases, like having multiple branches.

Reviewed By: RajivTS

Differential Revision: D48226070

fbshipit-source-id: eed970a8e4697ab10682e3b93863e6d621adaacc

Author: gustavoavena

eden/mononoke/Cargo.toml

@@ -282,6 +282,7 @@ members = [
   "git/git_types",
   "git/git_types/if",
   "git/git_types/if/types",
+  "git/gitexport",
   "git/gitimport",
   "git/import_direct",
   "git/import_tools",

eden/mononoke/git/gitexport/Cargo.toml

@@ -0,0 +1,28 @@
+# @generated by autocargo
+
+[package]
+name = "gitexport_tools"
+version = "0.1.0"
+authors = ["Facebook"]
+edition = "2021"
+license = "GPLv2+"
+
+[lib]
+path = "src/gitexport_tools/lib.rs"
+
+[[bin]]
+name = "gitexport"
+path = "src/main.rs"
+
+[dependencies]
+anyhow = "1.0.71"
+bookmarks_types = { version = "0.1.0", path = "../../bookmarks/bookmarks_types" }
+clap = { version = "4.3.5", features = ["derive", "env", "string", "unicode", "wrap_help"] }
+commit_graph = { version = "0.1.0", path = "../../repo_attributes/commit_graph/commit_graph" }
+fbinit = { version = "0.1.2", git = "https://github.com/facebookexperimental/rust-shed.git", branch = "main" }
+futures = { version = "0.3.28", features = ["async-await", "compat"] }
+in_memory_commit_graph_storage = { version = "0.1.0", path = "../../repo_attributes/commit_graph/in_memory_commit_graph_storage" }
+mononoke_api = { version = "0.1.0", path = "../../mononoke_api" }
+mononoke_app = { version = "0.1.0", path = "../../cmdlib/mononoke_app" }
+mononoke_types = { version = "0.1.0", path = "../../mononoke_types" }
+repo_authorization = { version = "0.1.0", path = "../../repo_authorization" }

eden/mononoke/git/gitexport/src/gitexport_tools/lib.rs

@@ -0,0 +1,99 @@
+/*
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This software may be used and distributed according to the terms of the
+ * GNU General Public License version 2.
+ */
+
+use std::sync::Arc;
+
+use commit_graph::CommitGraph;
+use futures::future::try_join_all;
+use futures::TryStreamExt;
+use in_memory_commit_graph_storage::InMemoryCommitGraphStorage;
+use mononoke_api::changeset_path::ChangesetPathHistoryContext;
+use mononoke_api::changeset_path::ChangesetPathHistoryOptions;
+pub use mononoke_api::BookmarkFreshness;
+use mononoke_api::ChangesetContext;
+use mononoke_api::MononokeError;
+use mononoke_api::MononokePath;
+use mononoke_types::RepositoryId;
+
+/// Given a list of paths and a changeset, return a commit graph
+/// containing only commits that are ancestors of the changeset and have
+/// modified at least one of the paths.
+pub async fn build_partial_commit_graph_for_export<P>(
+    paths: Vec<P>,
+    chgset_ctx: ChangesetContext,
+) -> Result<CommitGraph, MononokeError>
+where
+    P: TryInto<MononokePath>,
+    MononokeError: From<P::Error>,
+{
+    let mononoke_paths = paths
+        .into_iter()
+        .map(|path| path.try_into())
+        .collect::<Result<Vec<MononokePath>, _>>()?;
+
+    let chgset_path_hist_ctxs: Vec<ChangesetPathHistoryContext> = chgset_ctx
+        .paths_with_history(mononoke_paths.clone().into_iter())
+        .await?
+        .try_collect()
+        .await?;
+
+    // Get each path's history as a vector of changesets
+    let history_changesets: Vec<Vec<ChangesetContext>> = try_join_all(
+        try_join_all(
+            chgset_path_hist_ctxs
+                .iter()
+                // TODO(T160600443): support other ChangesetPathHistoryOptions
+                .map(|csphc| csphc.history(ChangesetPathHistoryOptions::default())),
+        )
+        .await?
+        .into_iter()
+        .map(|stream| stream.try_collect()),
+    )
+    .await?;
+
+    let sorted_changesets = merge_and_sort_changeset_lists(history_changesets)?;
+    let stripped_changesets = strip_irrelevant_changes(sorted_changesets, mononoke_paths).await?;
+
+    println!("sorted_changesets: {0:#?}", &stripped_changesets);
+
+    // TODO(gustavoavena): remove these prints for debugging after adding tests
+    let chgset_msgs: Vec<_> =
+        try_join_all(stripped_changesets.clone().iter().map(|csc| csc.message())).await?;
+    println!("chgset_msgs: {0:#?}", chgset_msgs);
+
+    create_commit_graph(stripped_changesets)
+}
+
+fn create_commit_graph(_changesets: Vec<ChangesetContext>) -> Result<CommitGraph, MononokeError> {
+    let cg_storage = Arc::new(InMemoryCommitGraphStorage::new(RepositoryId::new(1)));
+
+    let commit_graph = CommitGraph::new(cg_storage);
+
+    // TODO(T161204758): add commits to the commit graph
+
+    // TODO(T161204758): properly sort and dedupe the list of relevant changesets
+    Ok(commit_graph)
+}
+
+/// Given a list of changeset lists, merge, dedupe and sort them topologically
+/// into a single changeset list that can be used to build a commit graph.
+fn merge_and_sort_changeset_lists(
+    changesets: Vec<Vec<ChangesetContext>>,
+) -> Result<Vec<ChangesetContext>, MononokeError> {
+    // TODO(T161204758): properly sort and dedupe the list of relevant changesets
+    Ok(changesets.into_iter().flatten().collect())
+}
+
+/// Given a commit graph, create a new graph with every commit stripped of all
+/// changes that are not in any of the provided paths.
+async fn strip_irrelevant_changes(
+    changesets: Vec<ChangesetContext>,
+    _paths: Vec<MononokePath>,
+) -> Result<Vec<ChangesetContext>, MononokeError> {
+    // TODO(T161205476): strip irrelevant changes from a CommitGraph
+    Ok(changesets)
+}

eden/mononoke/git/gitexport/src/main.rs

@@ -0,0 +1,87 @@
+/*
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This software may be used and distributed according to the terms of the
+ * GNU General Public License version 2.
+ */
+
+use std::str::FromStr;
+
+use anyhow::Error;
+use bookmarks_types::BookmarkKey;
+use fbinit::FacebookInit;
+use gitexport_tools::build_partial_commit_graph_for_export;
+pub use mononoke_api::BookmarkFreshness;
+use mononoke_api::RepoContext;
+use mononoke_app::fb303::AliveService;
+use mononoke_app::fb303::Fb303AppExtension;
+use mononoke_app::MononokeApp;
+use mononoke_app::MononokeAppBuilder;
+use repo_authorization::AuthorizationContext;
+
+use crate::types::GitExportArgs;
+
+pub mod types {
+    use clap::Parser;
+    use mononoke_app::args::RepoArgs;
+
+    /// Mononoke Git Exporter
+    #[derive(Debug, Parser)]
+    pub struct GitExportArgs {
+        /// Name of the hg repo being exported
+        #[clap(flatten)]
+        pub hg_repo_args: RepoArgs,
+
+        /// Path to the git repo being created
+        #[clap(long)]
+        pub output: Option<String>, // TODO(T160787114): Make this required
+
+        /// List of directories in `hg_repo` to be exported to a git repo
+        #[clap(long)]
+        // TODO(T161204758): change this to a Vec<String> when we can support multiple export paths
+        pub export_path: String,
+        // TODO(T160600443): support last revision argument
+        // TODO(T160600443): support until_timestamp argument
+    }
+}
+
+#[fbinit::main]
+fn main(fb: FacebookInit) -> Result<(), Error> {
+    let app: MononokeApp = MononokeAppBuilder::new(fb)
+        .with_app_extension(Fb303AppExtension {})
+        .build::<GitExportArgs>()?;
+
+    app.run_with_monitoring_and_logging(async_main, "gitexport", AliveService)
+}
+
+async fn async_main(app: MononokeApp) -> Result<(), Error> {
+    let args: GitExportArgs = app.args()?;
+    let ctx = app.new_basic_context();
+
+    let repo = app.open_repo(&args.hg_repo_args).await?;
+
+    let auth_ctx = AuthorizationContext::new_bypass_access_control();
+    let repo_ctx: RepoContext = RepoContext::new(ctx, auth_ctx.into(), repo, None, None).await?;
+
+    // TODO(T160600443): support using a specific changeset as starting commit,
+    // instead of a bookmark.
+    let bookmark_key = BookmarkKey::from_str("master")?;
+
+    let chgset_ctx = repo_ctx
+        .resolve_bookmark(&bookmark_key, BookmarkFreshness::MostRecent)
+        .await?
+        .unwrap();
+
+    // TODO(T161204758): get multiple paths from arguments once we sort and
+    // dedupe changesets properly and can build a commit graph from multiple
+    // changeset lists
+    let export_paths = vec![args.export_path.as_str()];
+
+    let _commit_graph = build_partial_commit_graph_for_export(export_paths, chgset_ctx).await;
+
+    // TODO(T160787114): export commit graph to temporary mononore repo
+
+    // TODO(T160787114): export temporary repo to git repo
+
+    Ok(())
+}