Skip to content

Add change-id property to commit / refactor commit signing #3721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
May 15, 2024

Conversation

schacon
Copy link
Member

@schacon schacon commented May 7, 2024

This PR introduces a possibly controversial approach to how we're writing our our commit objects. In our move towards adding the ability to approach our workflows in a more "patch" oriented style, inspired by some of the Jujutsu tooling, we want to store a "change-id" in the commits that we write out. This will allow us to do complex rebasing-style operations more easily and allow users to refer to changes with an ID that doesn't change after a rebase (again, much like Jujutsu does).

The two ways we debated doing this were to do a Gerrit-like Git trailer or to write a new commit header when we write our commits out.

The trailer approach is less controversial in that it has been used in Gerrit tooling for a long time and is fully supported by all the normal Git tooling. However, it's pretty ugly and shows up everywhere in a way that could be confusing and unwanted.

What we're trying here is the header approach. This is a little more difficult, since libgit2 (and thus git-rs) has no easily supported way to write out custom headers other than the commit_signed method (which technically takes a variable header name): https://docs.rs/git2/latest/git2/struct.Repository.html#method.commit_signed. However, it's doable by creating a commit buffer in the same way, then injecting our own new header before the \n\n message delimiter, the same way that commit_signed does with the signature data. Then we just write this buffer as an object directly with ODB::write (https://docs.rs/git2/latest/git2/struct.Odb.html#method.write).

In testing, we found that git-fsck doesn't complain about unknown headers, rebase and cherry-pick will retain them, log will not show them unless you specify --pretty=raw, etc. Git just doesn't seem to care at all about extra headers as long as they're properly formed. They're not really supported officially, and I haven't tested issues with other implementations like JGit, but libgit2 and Git itself do not seem to care about them in any way.

Which means we can use them to store "hidden" information like this change-id, and it's easy to extract and use the data.

So, in this PR, each commit has a change-id added to it's header field so we can more easily track patches by unique ids.

It also introduces a total change in the way that commit signing works, since the move towards rewriting commits means we almost always lose our signatures (rebasing currently doesn't re-write the signatures). It moves from native SSH-key signing (limited to our own key and controlled via our settings) to fork/exec'ing the gpg/ssh binaries to sign the commit data, the same way Git itself does.

This also removes the signing setting, we just depend on Git's config values (commit.gpgSign, etc) to determine if we try to sign stuff.

@schacon schacon force-pushed the add-commit-id branch 3 times, most recently from 6f37888 to 206d19c Compare May 7, 2024 08:28
Each commit has a change-id added to it's header field so we can more easily track patches by unique ids
@schacon schacon force-pushed the add-commit-id branch 6 times, most recently from 7837376 to fbedccc Compare May 7, 2024 13:15
in the strangest possible way
@schacon schacon force-pushed the add-commit-id branch 2 times, most recently from 3f9fe26 to 4d54233 Compare May 11, 2024 09:48
@schacon schacon marked this pull request as ready for review May 14, 2024 11:17
@schacon
Copy link
Member Author

schacon commented May 15, 2024

OK, I have also verified that both gpg and ssh signing work on Windows too.

The logic for signing commits is now to check for commit.gpgSign being set. If it's set, then we look to see if we have a signing key in user.signingkey. If we have a key, we look for 'ssh' in gpg.format, otherwise we use GPG. We will respect gpg.ssh.program for ssh if there is a different binary path, and gpg.program for GPG. We also identify literal SSH keys in the user.signingkey field.

The only major thing we don't support yet is gpg.ssh.defaultKeyCommand for other ways to get a key other than the user.signingkey field. We also don't support the X.509 smime stuff.

This will break ssh signing for people who used our in-built stuff, but it should be fairly simple to set it back up again with some Git commands. Namely something like:

$ git config --global user.signingkey "~/Library/Application Support/com.gitbutler.app/keys/ed25519.pub"
$ git config --global gpg.format ssh
$ git config --global commit.gpgsign true

@schacon
Copy link
Member Author

schacon commented May 15, 2024

I can also confirm that JGit does not seem to care about the extra header. That's the only other major implementation that I know that most of the other tools are built on, so I assume nothing has an issue with this extra header. It logically makes sense, as there are a few other uncommon supported headers (encoding, gpgsig-sha256, etc) so everything should be designed to just ignore unknown commit headers.

@@ -321,6 +324,7 @@ fn verify_head_is_clean(
)
.context("failed to reset to integration commit")?;

dbg!("Head creating virtual branch");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

Comment on lines 39 to 46
serde_json = { version = "1.0", features = [ "std", "arbitrary_precision" ] }
serde_json = { version = "1.0", features = ["std", "arbitrary_precision"] }
sha2 = "0.10.8"
ssh-key = { version = "0.6.6", features = [ "alloc", "ed25519" ] }
ssh-key = { version = "0.6.6", features = ["alloc", "ed25519"] }
ssh2 = { version = "0.9.4", features = ["vendored-openssl"] }
strum = { version = "0.26", features = ["derive"] }
log = "^0.4"
tempfile = "3.10"
thiserror.workspace = true
tokio = { workspace = true, features = [ "rt-multi-thread", "rt", "macros" ] }
tokio = { workspace = true, features = ["rt-multi-thread", "rt", "macros"] }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated but it seems we don't have a prettier setup for formatting toml. Are these space changes intended?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I had some prettier formatter running on the toml file, so it fixed them. I reverted.

@schacon schacon enabled auto-merge May 15, 2024 13:25
@schacon schacon changed the title Add change-id property to commit Add change-id property to commit / refactor commit signing May 15, 2024
@schacon schacon merged commit 4b68382 into master May 15, 2024
@schacon schacon deleted the add-commit-id branch May 15, 2024 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants