Preliminary fixes for reftable support #669

hanwen · 2020-06-30T10:30:29Z

These are assorted small fixes split out from the reftable topic.

hanwen · 2020-06-30T12:04:46Z

/submit

gitgitgadget · 2020-06-30T12:06:15Z

Submitted as pull.669.git.1593518738.gitgitgadget@gmail.com

gitgitgadget · 2020-07-07T04:41:50Z

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> These are assorted small fixes split out from the reftable topic.
>
> Han-Wen Nienhuys (4):
>   lib-t6000.sh: write tag using git-update-ref
>   t3432: use git-reflog to inspect the reflog for HEAD
>   checkout: add '\n' to reflog message
>   Treat BISECT_HEAD as a pseudo ref
>
>  builtin/bisect--helper.c       | 3 +--
>  builtin/checkout.c             | 5 +++--
>  git-bisect.sh                  | 4 ++--
>  t/lib-t6000.sh                 | 5 ++---
>  t/t3432-rebase-fast-forward.sh | 7 ++++---
>  5 files changed, 12 insertions(+), 12 deletions(-)

Thanks for splitting these out, and sorry for not being able to get
to them earlier.

Will queue.

gitgitgadget · 2020-07-07T05:01:05Z

This branch is now known as hn/reftable.

gitgitgadget · 2020-07-07T05:01:06Z

This patch series was integrated into seen via git@fd5c471.

gitgitgadget · 2020-07-07T05:58:27Z

This patch series was integrated into seen via git@66f3a24.

gitgitgadget · 2020-07-07T21:17:39Z

This patch series was integrated into seen via git@da40f20.

gitgitgadget · 2020-07-07T22:32:51Z

This patch series was integrated into seen via git@3dfc36e.

gitgitgadget · 2020-07-08T22:43:03Z

This patch series was integrated into seen via git@f1c2194.

gitgitgadget · 2020-07-09T00:03:51Z

This patch series was integrated into seen via git@aefc3ef.

gitgitgadget · 2020-07-09T02:06:42Z

This patch series was integrated into seen via git@6f7cfa5.

gitgitgadget · 2020-07-09T05:42:00Z

This patch series was integrated into seen via git@2cb0ca1.

gitgitgadget · 2020-07-09T19:52:21Z

This patch series was integrated into seen via git@dfe6f6e.

gitgitgadget · 2020-07-09T20:45:47Z

This patch series was integrated into seen via git@98dd40e.

gitgitgadget · 2020-07-09T21:46:24Z

This patch series was integrated into seen via git@17af751.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

Both the git-bisect.sh as bisect--helper inspected the file system directly. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

Regarding reflog messages: - We expect that a reflog message consists of a single line. The file format used by the files backend may add a LF after the message as a delimiter, and output by commands like "git log -g" may complete such an incomplete line by adding a LF at the end, but philosophically, the terminating LF is not a part of the message. - We however allow callers of refs API to supply a random sequence of NUL terminated bytes. We cleanse caller-supplied message by squashing a run of whitespaces into a SP, and by trimming trailing whitespace, before storing the message. This is how we tolerate, instead of erring out, a message with LF in it (be it at the end, in the middle, or both). Currently, the cleansing of the reflog message is done by the files backend, before the log is written out. This is sufficient with the current code, as that is the only backend that writes reflogs. But new backends can be added that write reflogs, and we'd want the resulting log message we would read out of "log -g" the same no matter what backend is used. An added benefit is that the "cleansing" function could be updated later, independent from individual backends, to e.g. allow multi-line log messages if we wanted to, and when that happens, it would help a lot to ensure we covered all bases if the cleansing function (which would be updated) is called from the generic layer. Side note: I am not interested in supporting multi-line reflog messages right at the moment (nobody is asking for it), but I envision that instead of the "squash a run of whitespaces into a SP and rtrim" cleansing, we can %urlencode problematic bytes in the message *AND* append a SP at the end, when a new version of Git that supports multi-line and/or verbatim reflog messages writes a reflog record. The reading side can detect the presense of SP at the end (which should have been rtrimmed out if it were written by existing versions of Git) as a signal that decoding %urlencode recovers the original reflog message. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

hanwen · 2020-07-10T17:19:09Z

/submit

gitgitgadget · 2020-07-10T17:20:19Z

Submitted as pull.669.v2.git.1594401593.gitgitgadget@gmail.com

gitgitgadget · 2020-07-10T21:00:32Z

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> These are assorted small fixes split out from the reftable topic.

Will replace.  I'll retitle the 3/4 and 4/4 to

    bisect: treat BISECT_HEAD as a pseudo ref
    reflgo: cleanse messages in the refs.c layer

while queuing.

Thanks.

> Han-Wen Nienhuys (3):
>   lib-t6000.sh: write tag using git-update-ref
>   t3432: use git-reflog to inspect the reflog for HEAD
>   Treat BISECT_HEAD as a pseudo ref
>
> Junio C Hamano (1):
>   Cleanse reflog messages in the refs.c layer
>
>  builtin/bisect--helper.c       |  3 +-
>  git-bisect.sh                  |  4 +--
>  refs.c                         | 50 ++++++++++++++++++++++++++++------
>  refs/files-backend.c           |  2 +-
>  refs/refs-internal.h           |  6 ----
>  t/lib-t6000.sh                 |  5 ++--
>  t/t3432-rebase-fast-forward.sh |  7 +++--
>  7 files changed, 51 insertions(+), 26 deletions(-)
>
>
> base-commit: bd42bbe1a46c0fe486fc33e82969275e27e4dc19
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-669%2Fhanwen%2Fpreliminaries-v2
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-669/hanwen/preliminaries-v2
> Pull-Request: https://github.com/gitgitgadget/git/pull/669
>
> Range-diff vs v1:
>
>  1:  b7b2ad8e79 = 1:  0f244ed6cb lib-t6000.sh: write tag using git-update-ref
>  2:  f238d1d7f8 ! 2:  123d246edf t3432: use git-reflog to inspect the reflog for HEAD
>      @@ t/t3432-rebase-fast-forward.sh: test_rebase_same_head_ () {
>        		oldhead=\$(git rev-parse HEAD) &&
>        		test_when_finished 'git reset --hard \$oldhead' &&
>       -		cp .git/logs/HEAD expect &&
>      -+		git reflog HEAD > expect &&
>      ++		git reflog HEAD >expect &&
>        		git rebase$flag $* >stdout &&
>      -+		git reflog HEAD > actual &&
>      ++		git reflog HEAD >actual &&
>        		if test $what = work
>        		then
>        			old=\$(wc -l <expect) &&
>  3:  8a62cc2668 < -:  ---------- checkout: add '\n' to reflog message
>  4:  2b7eb58c15 = 3:  d4007c2a5b Treat BISECT_HEAD as a pseudo ref
>  -:  ---------- > 4:  6ca5b99c8d Cleanse reflog messages in the refs.c layer

gitgitgadget · 2020-07-10T21:09:31Z

refs.c

@@ -902,7 +902,7 @@ int delete_ref(const char *msg, const char *refname,
 			       old_oid, flags);


On the Git mailing list, Junio C Hamano wrote (reply to this):

"Junio C Hamano via GitGitGadget" <gitgitgadget@gmail.com> writes: > Currently, the cleansing of the reflog message is done by the files > backend, before the log is written out. This is sufficient with the > current code, as that is the only backend that writes reflogs. But > new backends can be added that write reflogs, and we'd want the > resulting log message we would read out of "log -g" the same no > matter what backend is used. I _think_ there is no correctness impact, but this slightly changes the semantics. We used to take whatever random string from the caller and at the lowest layer of the files backend, i.e. > strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); > if (msg && *msg) > - copy_reflog_msg(&sb, msg); > + strbuf_addstr(&sb, msg); cleansed and added the message to strbuf. With the updated code, normalize_reflog_message() is defined like so > +static char *normalize_reflog_message(const char *msg) > +{ > + struct strbuf sb = STRBUF_INIT; > + > + if (msg && *msg) > + copy_reflog_msg(&sb, msg); > + return strbuf_detach(&sb, NULL); > +} to always return a non-NULL but an empty string, even if the caller gave us a NULL pointer. We always pass a non-NULL string around, and the updated low-level code avoids calling strbuf_addstr() because msg is not NULL, but *msg is '\0'. We _might_ be able to optimize by tweaking the normalizer a bit further, perhaps like so: static char *normalize_reflog_message(const char *msg) { struct strbuf sb = STRBUF_INIT; if (msg && *msg) copy_reflog_msg(&sb, msg); if (sb.len) { return strbuf_detach(&sb, NULL); } else { strbuf_release(&sb); return NULL; } } to avoid allocating 1 byte '\0' on the heap by strbuf_detach() when sb here ends up being truly empty. But it would be a premature optimization to worry about such a thing at this point, I think. Thanks.

gitgitgadget · 2020-07-10T22:51:24Z

This patch series was integrated into seen via git@5a80cc8.

gitgitgadget · 2020-07-13T22:46:59Z

This patch series was integrated into seen via git@30eb2cc.

gitgitgadget · 2020-07-14T22:00:58Z

This patch series was integrated into seen via git@9a5a3af.

gitgitgadget · 2020-07-14T22:00:59Z

This patch series was integrated into next via git@e524883.

gitgitgadget · 2020-07-15T18:24:40Z

This patch series was integrated into seen via git@0c929c6.

gitgitgadget · 2020-07-15T22:38:41Z

This patch series was integrated into seen via git@430470b.

gitgitgadget · 2020-07-16T00:36:15Z

This patch series was integrated into seen via git@39c3065.

gitgitgadget · 2020-07-16T21:19:37Z

This patch series was integrated into seen via git@5cefbac.

gitgitgadget · 2020-07-16T23:14:43Z

This patch series was integrated into seen via git@29b9f5e.

gitgitgadget · 2020-07-17T00:55:09Z

This patch series was integrated into seen via git@61d58e8.

gitgitgadget · 2020-07-17T20:25:44Z

This patch series was integrated into seen via git@d0f73d8.

gitgitgadget · 2020-07-18T23:31:56Z

This patch series was integrated into seen via git@3428f9c.

gitgitgadget · 2020-07-19T01:24:23Z

This patch series was integrated into seen via git@5e2a57a.

gitgitgadget · 2020-07-20T21:18:45Z

This patch series was integrated into seen via git@a60fefe.

gitgitgadget · 2020-07-21T05:19:47Z

This patch series was integrated into seen via git@1c6a945.

gitgitgadget · 2020-07-21T22:12:18Z

This patch series was integrated into seen via git@2102555.

gitgitgadget · 2020-07-22T17:36:48Z

This patch series was integrated into seen via git@9f1be6f.

gitgitgadget · 2020-07-22T22:43:04Z

This patch series was integrated into seen via git@b590dc8.

gitgitgadget · 2020-07-24T21:38:48Z

This patch series was integrated into seen via git@71fd332.

gitgitgadget · 2020-07-24T23:24:24Z

This patch series was integrated into seen via git@902db90.

gitgitgadget · 2020-07-26T20:21:33Z

This patch series was integrated into seen via git@5aa6db6.

gitgitgadget · 2020-07-27T16:53:57Z

This patch series was integrated into seen via git@f413529.

gitgitgadget · 2020-07-30T21:39:14Z

This patch series was integrated into seen via git@3161cc6.

gitgitgadget · 2020-07-30T21:39:15Z

This patch series was integrated into next via git@3161cc6.

gitgitgadget · 2020-07-30T21:39:16Z

This patch series was integrated into master via git@3161cc6.

gitgitgadget · 2020-07-30T21:39:19Z

Closed via 3161cc6.

weekly-digest bot mentioned this pull request Jul 5, 2020

Weekly Digest (28 June, 2020 - 5 July, 2020) #672

Closed

gitgitgadget bot added the seen label Jul 7, 2020

hanwen and others added 4 commits July 10, 2020 18:39

lib-t6000.sh: write tag using git-update-ref

0f244ed

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

t3432: use git-reflog to inspect the reflog for HEAD

123d246

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

Treat BISECT_HEAD as a pseudo ref

d4007c2

Both the git-bisect.sh as bisect--helper inspected the file system directly. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

gitgitgadget bot reviewed Jul 10, 2020

View reviewed changes

gitgitgadget bot added the next label Jul 14, 2020

This was referenced Jul 26, 2020

Weekly Digest (19 July, 2020 - 26 July, 2020) #682

Closed

Weekly Digest (19 July, 2020 - 26 July, 2020) #683

Closed

gitgitgadget bot added the master label Jul 30, 2020

gitgitgadget bot closed this Jul 30, 2020

		@@ -902,7 +902,7 @@ int delete_ref(const char msg, const char refname,
		old_oid, flags);

Preliminary fixes for reftable support #669

Preliminary fixes for reftable support #669

Uh oh!

Conversation

hanwen commented Jun 30, 2020

Uh oh!

hanwen commented Jun 30, 2020

Uh oh!

gitgitgadget bot commented Jun 30, 2020

Uh oh!

gitgitgadget bot commented Jul 7, 2020

Uh oh!

gitgitgadget bot commented Jul 7, 2020

Uh oh!

gitgitgadget bot commented Jul 7, 2020

Uh oh!

gitgitgadget bot commented Jul 7, 2020

Uh oh!

gitgitgadget bot commented Jul 7, 2020

Uh oh!

gitgitgadget bot commented Jul 7, 2020

Uh oh!

gitgitgadget bot commented Jul 8, 2020

Uh oh!

gitgitgadget bot commented Jul 9, 2020

Uh oh!

gitgitgadget bot commented Jul 9, 2020

Uh oh!

gitgitgadget bot commented Jul 9, 2020

Uh oh!

gitgitgadget bot commented Jul 9, 2020

Uh oh!

gitgitgadget bot commented Jul 9, 2020

Uh oh!

gitgitgadget bot commented Jul 9, 2020

Uh oh!

hanwen commented Jul 10, 2020

Uh oh!

gitgitgadget bot commented Jul 10, 2020

Uh oh!

gitgitgadget bot commented Jul 10, 2020

Uh oh!

gitgitgadget bot Jul 10, 2020

Choose a reason for hiding this comment

Uh oh!

gitgitgadget bot commented Jul 10, 2020

Uh oh!

gitgitgadget bot commented Jul 13, 2020

Uh oh!

gitgitgadget bot commented Jul 14, 2020

Uh oh!

gitgitgadget bot commented Jul 14, 2020

Uh oh!

gitgitgadget bot commented Jul 15, 2020

Uh oh!

gitgitgadget bot commented Jul 15, 2020

Uh oh!

gitgitgadget bot commented Jul 16, 2020

Uh oh!

gitgitgadget bot commented Jul 16, 2020

Uh oh!

gitgitgadget bot commented Jul 16, 2020

Uh oh!

gitgitgadget bot commented Jul 17, 2020

Uh oh!

gitgitgadget bot commented Jul 17, 2020

Uh oh!

gitgitgadget bot commented Jul 18, 2020

Uh oh!

gitgitgadget bot commented Jul 19, 2020

Uh oh!

gitgitgadget bot commented Jul 20, 2020

Uh oh!

gitgitgadget bot commented Jul 21, 2020

Uh oh!

gitgitgadget bot commented Jul 21, 2020

Uh oh!

gitgitgadget bot commented Jul 22, 2020

Uh oh!

gitgitgadget bot commented Jul 22, 2020