New branch creation causes the plugin to iterate over all changesets

[itay]

We have a very large repo (over 100K changes, 10 years of history). If we enable the plugin, then pushing a new branch to the server basically causes the push to hang and eventually die, because it will check every commit since the beginning of time.

The above happens because Stash gives you a from hash of 000000 and a to hash of <sha>, which will give you all changesets when you do the iteration.

There's no good fix, but a good medium term fix will be to just add a limit to how many changes are requested for a new branch push (e.g. 100).

[christiangalsterer]

I will look into this the next days.

[rogueluke]

This seems to also happen when new tags are pushed to the server

[gjoseph]

Hey Christian, any news about this ?

[christiangalsterer]

I was on vacation and just returned. I will first concentrate on making the plugin compatible with Bitbucket 4.x (#5). Shortly after that I will look into this improvement. In order to know on which version to concentrate first it would be good to know which versions of Bitbucket you are running and/or if you plan to update to Bitbucket in the near future, so please leave a comment with the version information for Bitbucket.

[gjoseph]

We're currently on Stash v3.11.0, but we'll probably upgrade to BBServer soon. (in fact, I hadn't paid attention to news, and it's your message above that made me aware of the rebranding)

[christiangalsterer]

I checked on the Bitbucket API, but I'm not sure yet how to implement this.
Detecting a new branch should be possible but how to limit the number of commits is not clear yet. If somebody has an idea, please share. Will keep on searching.

[gjoseph]

IIRC in the case of new branches the "from" hash was 00000000, which is (again, IIRC) how we detected these cases in our gitolite hooks. The bash snippet to find the revlist to check was like this:

while read before after branch; do                                       # For each commit...
  # echo "$0: testing before='$before' after='$after' branch='$branch'"
  if ! [[ $after =~ ^0+$ ]]; then

    if [[ $before =~ ^0+$ ]]; then
      revlist="$(git rev-list "$after" --not --branches=*)"
    else
      revlist="$(git rev-list "$before..$after")"
    fi

[grumbelbart]

It seems that the bitbucket API has no way of running

git rev-list $new_commit --not --branches=*

If I understand this correctly. Some workaround to get only those commits that are really "new" to the object store would be required.

[grumbelbart]

Apparently, there is a way of filtering those commits which are new to the repository: https://answers.atlassian.com/questions/223143/how-to-get-only-new-changesets-in-pre-receive-hook-on-new-branch-push

[tarnowsc]

I've run into the same performance issue and updated the plugin to use diff-tree. Now on repo with >50k commits and >15k files the response is instant, whereas earlier I was getting timeouts.

Those changes are available here #26.

[christiangalsterer]

Thank you very much for the PR.
Currently I'm travelling, but I will look into it in roughly one week.

[tarnowsc]

I've found a limitation on using the diff-tree, it squashes the delta and omits files that were both added and removed in the range of commits.

My case is to clean up a repository with 5 years of history from binary files and to protect it from further degradation. Unfortunately it takes just hours for people to pollute it with branches containing old history and it makes whole cleanup pointless.

I was trying to modify the diff-tree call a bit unfortunately with no luck.

Now I'm looking into using git-log ...

But maybe there is some better way?

[mithun]

Bitbucket Server 5.0 introduced a new ADDED_TO_REPOSITORY filter that can be used to filter and examine only new commits that will be added to the repository