oxidize Git Cherry Tree

[Special Bread (special-bread)]

Just following up on #2712 (reply in thread)

First I wanted to say thanks for making gix, it is very pleasing, and helped get a lot of stuff working well in Git Cherry Tree :>

Generally speaking, the main things blocking git cherry tree from being fully on gix is ways of getting confidence that my code really does always work. Libgit2 seems to be fairly easy to use for an initial implementation, and I haven't had that many issues with a naive implementation. My experience with gix is that I need to become an expert in that part of the git world in order to get the results I need and not run into bugs.

For context, Git Cherry Tree uses libgit2, and gix. It doesnt need the git CLI at all to run, but does need git LFS installed if you want to work with LFS repos. It also runs without any installer or any github/etc accounts.

That said, aside from that, It seems like there is not that much blocking!

Whats blocking

Nice push. It seems that there is only the default config there. As long as its safe, reasonable speed, that would be enough.

• Git Cherry Tree supports multiple git accounts/credentials so control over which one gets used would be good.
• sendpack over smart https, and resuing the fetch credential systems for that
• refspec matching on push
• force-with-lease support - i.e. expected old OID and compare and swap stuff
• delta compression for pack generation - I believe currently only full objects are supported, without that pushed packs from a client that was comitting locally cause issues
• report-status (-v2) per ref, so that rejection reasons can be shown per branch

For context: Git Cherry Tree only has push and fetch as remote operations, where "pull" is a local only operation (no surprise pulls into who knows what remote allowed!). Pushing tries to pick credentials that match a given repo, and force pushing is always with lease. It also tries to let the user know nicely why a push may have failed.

GCT also checks if the outgoing stuff contains any LFS pointers and if it does it runs git lfs push first through the git-lfs binary before the ref update. So gix would only need to be git objects.

Not blocking but would be nice to have

• Post merge conflict resolution: Given a conflicted index, resolve a path to ours/theirs/union, and set to stage 0 and write, and the plumbing associated with that. Possibly also more detailed stuff like outputting the result to an in memory data structure instead of writing to disk for in memory resolution (particularly when doing so without anything checked out in the first place).
• A bulk write path to the ODB. On windows, writing loose objects is very slow! Committing the linux repo (90k files, 1-2gb) takes 16 minutes on the git CLI! Git Cherry Tree uses gix (nice!) to be several times faster: 1-2minutes, and 4-5 seconds if the objects are already there. Much of that is spent on exists(), and also each miss hits consolidate_with_disk_state which happens by default on writing new blobs, so its typically hitting that. Looking at this from calculations, it should be something like a few seconds to commit that amount, so there is lots of improvent left. (after you mentioned commits seem slow, I checked and they are indeed much slower than they should be! But as you can see windows is very slow also!)
• Fetch prune would be nice, I believe the fields are commented out for that.
• Hooks - libgit2 never had hooks, so this doesnt block or slow switching to gix. but would be pleasing indeed if gix had hooks!
• Sequencer state writes and reads in some nice API. I need to get rebases/etc read from and checked out to disk to be able to resume them in memory or vice versa, and so this would be nice to handle.

Also nice to have but already handled

Here is stuff GCT already did and ran into, so this wouldn't help but worth knowing for others who might run into it:

• Index writes that handle the tree extension. Right now its dropped in Git Cherry Tree on the index write, but would be nice if it worked by default without needing the caller to know anything, and also updated the tree extension correctly!
• Checkout a tree - GCT uses a gix for this, custom written with a multithreaded checkout of the tree blobs and such. This is used for whenever anything at all needs checking out including switching branches, rebases, merges, rename/split/squash, pull, reset refs, undo/redo etc. This already works, but a higher level version on gix would be pleasing though.
• topological order with first parent and hiding/boundaries being reachable through Repository::rev_walk() as opposed to the lower level gix::traverse
• GCT has a custom status implementation, which is very fast and runs on gix :>
• GCT uses gix for attributes and LFS classification, and has some additional stuff on top to do things like dereference pointers, check for lfs stuff before pushing, identify broken lfs pointers/missing blobs, etc.

Also a potential bug report:
A filter driver config like [filter "lfs"] in a global git config overrides a repo-local config, which is backwards from git. I think i also had an issue where a broken driver fails closed even with required=false as opposed to git passing through raw content, but not sure about that one. I have a repro repo on my machine with an issue to write up here, putting that here to poke you into poking me :> I ran into some of this stuff while working on LFS support.

Stuff Git Cherry Tree doesn't do

• staging/unstaging of any kind :>
• git blame. Maybe it will later!
• absolutely no interactive rebase! The whole client is interactive and can rebase!

Hope this helps!

[Sebastian Thiel (Byron)]

Thanks so much, it's great to see you (and GCT) here!

First off, I think a lot of progress will be made this year, and in the year to follow, so I think we will get everything sorted out.

Overall, I think a few of the problems you are having shouldn't (still) be problems, but maybe this ties into this:

My experience with gix is that I need to become an expert in that part of the git world in order to get the results I need and not run into bugs.

That's kind of the opposite experience that I wish gix users to have 😅, even if I generously replace "bugs" with "surprises". When using gix::Repository everything should be reasonably simple, but without hiding certain complexities if they would help make things faster or allow users to control their memory consumption. But again, I think we will get there.

I think i also had an issue where a broken driver fails closed even with required=false as opposed to git passing through raw content

This should already be fixed by now, and I am looking into the incorrect filter configuration problem.

topological order with first parent and hiding/boundaries being reachable through Repository::rev_walk()

It can hide now, so I'd think it should work for you.

Checkout a tree

Yes, that will definitely be coming. Something that you'd need is a way to have a programatic filter inserted into the pipeline (a custom LFS implementation). Actually I planned to ship gix with LFS support as well, but it would be easier to support programmatic filters to start with. This would probably be a big deal for you, as you could use worktree filter pipelines naturally and just have to deal with figuring out what to checkout, and how to do it safely.

Index writes that handle the tree extension

This will also happen sooner than later.

Post merge conflict resolution

While this is already possible by manipulating the index directly, it's probably something that can benefit from an abstraction layer on top. I will probably get there eventually.

A bulk write path to the ODB.

This exists. Enable in-memory objects on the Repository handle, then take out the memory and write the contained objects in any way you like. There is even new methods that take an existing ID as input so it won't be hashed again.

Hooks

Will happen, it's planned.

Sequencer state writes and reads

I agree, there already is a gix-sequencer crate, but it's empty.

Git Cherry Tree supports multiple git accounts

This should already work today as you can pass a programmatic credentials implementation, so auth-requests go to you in case of HTTP. SSH is fully under control of ssh though, just like in Git.

[my own words] push push push

Yes, definitely, it's planned, but might be a while.

report-status (-v2) per ref, so that rejection reasons can be shown per branch

I am pulling a blank here, can you elaborate?

staging/unstaging of any kind :>

The index really has an unnecessarily bad rep :D. However, I'd also hide it if I can, but still make sure it's handled correctly, something that I think you already do.

I hope this covers all points, roughly, and I am sure in a year from now we will both be in a much better spot when it comes to supporting all Git operations natively. There is a lot of work in progress that needs review and tuning, and each of these will make substantial improvements.

Also I plan to implement my very own Git client, which will certainly push me to make various improvements to gix along the way as well.

[Sebastian Thiel (Byron)]

Oh, and before I forget: the .exists() fall through will also be remedied soon. A big PR is waiting for review, for it to work reasonably out of the box. Meantime, you probably want to set it to Refresh::Never.