tkldev: Weird behaviour in common when using git

[JedMeister]

I'm pretty sure that this is a bug in tkldev-setup on Debian Bullseye. Although I'm not clear how/why... Perhaps it's because they are shallow clones by default? Perhaps we should rethink that?

Basically the common repo is acting really weird in TKLDev v17.0beta1.

E.g. (I was trying to checkout the 17.x-dev branch):

root@tkldev .../products/core# cd common/
/turnkey/fab/common
root@tkldev .../fab/common# gb
* master                cb046a0 Merge pull request #196 from JedMeister/tweak-apt
  remotes/origin/HEAD   -> origin/master
  remotes/origin/master cb046a0 Merge pull request #196 from JedMeister/tweak-apt
root@tkldev .../fab/common# git fetch origin 
root@tkldev .../fab/common# gb
* master                cb046a0 Merge pull request #196 from JedMeister/tweak-apt
  remotes/origin/HEAD   -> origin/master
  remotes/origin/master cb046a0 Merge pull request #196 from JedMeister/tweak-apt
root@tkldev .../fab/common# git remote -v
origin	https://github.com/turnkeylinux/common (fetch)
origin	https://github.com/turnkeylinux/common (push)
root@tkldev .../fab/common# git checkout 17.x-dev
error: pathspec '17.x-dev' did not match any file(s) known to git
root@tkldev .../fab/common# git checkout origin/17.x-dev
error: pathspec 'origin/17.x-dev' did not match any file(s) known to git
root@tkldev .../fab/common# git fetch origin 
root@tkldev .../fab/common# git branch -r
  origin/HEAD -> origin/master
  origin/master
root@tkldev .../fab/common# git remote update
Fetching origin
root@tkldev .../fab/common# git fetch --all  
Fetching origin

I did discover a workaround:

root@tkldev .../fab/common# git ls-remote origin
cb046a097128b9461b8d42a27d3f22e9f26eeba9	HEAD
de4d09397bbc342a090f05744852ffe9ae3b9102	refs/heads/15.x
cb046a097128b9461b8d42a27d3f22e9f26eeba9	refs/heads/16.x
fbc9cd89d9002d2c0a53a0a6ad2675dd2ffb9b2a	refs/heads/17.x-dev
cb046a097128b9461b8d42a27d3f22e9f26eeba9	refs/heads/master
15c8d065ab385f6d38f53095f39dd953e0f21367	refs/pull/1/head
40e062225d9e49c17e1eee517b1167a869e00b5e	refs/pull/1/merge
6d45dd1d91584a72d14c7cd5a3759818aadf62e4	refs/pull/10/head
a2c612c04e1bef0b956cab92b13bebf8f18a520d	refs/pull/10/merge
10b8020dd70d0733eb01c0ef916906431571c88a	refs/pull/100/head
373ed068cf2ac7e5611ea1c759f120b6fd316d06	refs/pull/101/head
<snip></snip>
843e7d65b97e73e1771a008a4a6030761097e072	refs/pull/96/head
36363577a4f50f35b443dc45c61c95c989ef7bc9	refs/pull/97/head
78396d3b71617160a035554d548c642c59dea946	refs/pull/98/head
b56abf04b4c5433a225db4d9593a10042e7935e4	refs/pull/99/head
745f3bd5d7b20a1c8dd5b6cf985e318ebffdf6af	refs/tags/11.3
8a9cdb4c93dc52ddf9167ae9d846d2f0483f803e	refs/tags/12.0
d7e8a23e5f0bb4972397f6333be2a84a23eb7c4d	refs/tags/12.1
0a5e79202b5c1e33de864fb3c3076bbb7b92eb76	refs/tags/13.0
1c403fea7cd9b109ac5d07c502a931021d2db889	refs/tags/13.0^{}
8e882fe6d819d36100d5da0802a9aaee22b8a983	refs/tags/14.0
e11df989c92ef7e7c63b54caaba01f8f6ea58d01	refs/tags/14.0^{}
1aaf3a234ee990c75d940b229f3a80d4e5191602	refs/tags/14.1
a57f65f33a6dd44d75b03cf95f4a167c47c4ba3c	refs/tags/14.1^{}
81fcf2f77fc6b310056f9548a9317a91e7efc8a8	refs/tags/14.2
6bd3712df39dc316e7e30ad828087ed49631b9fa	refs/tags/14.2^{}
52af77d7d0c5ce1ef31d99e338f4ae5430f91c27	refs/tags/15.x
de4d09397bbc342a090f05744852ffe9ae3b9102	refs/tags/15.x^{}
e27f8cf03ac0e3f1a24849a14d0b701e6abb249f	refs/tags/16.x
8f432ecc8d76bd149c837c007dfaf53f9aaf4b3d	refs/tags/16.x^{}
root@tkldev .../fab/common# git fetch origin 17.x-dev:17.x-dev
remote: Enumerating objects: 6979, done.
remote: Counting objects: 100% (6977/6977), done.
remote: Compressing objects: 100% (2742/2742), done.
remote: Total 6648 (delta 2923), reused 6424 (delta 2717), pack-reused 0
Receiving objects: 100% (6648/6648), 4.29 MiB | 9.92 MiB/s, done.
Resolving deltas: 100% (2923/2923), completed with 130 local objects.
From https://github.com/turnkeylinux/common
 * [new branch]      17.x-dev   -> 17.x-dev
 * [new tag]         11.3       -> 11.3
 * [new tag]         12.0       -> 12.0
 * [new tag]         12.1       -> 12.1
 * [new tag]         13.0       -> 13.0
 * [new tag]         14.0       -> 14.0
 * [new tag]         14.1       -> 14.1
 * [new tag]         14.2       -> 14.2
 * [new tag]         15.x       -> 15.x
 * [new tag]         16.x       -> 16.x

But even after that, I still can't see any branch other than master and 17.x-dev (there should also be 16.x & 15.x). The "fix" is to remove the remote and add it back in:

root@tkldev .../fab/common# git remote rm origin
root@tkldev .../fab/common# git remote add origin https://github.com/turnkeylinux/common
root@tkldev .../fab/common# git fetch origin
From https://github.com/turnkeylinux/common
 * [new branch]      15.x       -> origin/15.x
 * [new branch]      16.x       -> origin/16.x
 * [new branch]      17.x-dev   -> origin/17.x-dev
 * [new branch]      master     -> origin/master

See proper workaround below.

[JedMeister]

It turns out that this is a result of tkldev-setup doing shallow clones.

The proper workaround is to set remote branches to '*'. I.e.:

git remote set-branches origin '*'

E.g. in the case of common (I've added an additional line break between each command to make it easier to read):

root@tkldev ~# cd common
/turnkey/fab/common

root@tkldev .../fab/common# gb # git branch -av
* master                cb046a0 Merge pull request #196 from JedMeister/tweak-apt
  remotes/origin/HEAD   -> origin/master
  remotes/origin/master cb046a0 Merge pull request #196 from JedMeister/tweak-apt

root@tkldev .../fab/common# git fetch origin 

root@tkldev .../fab/common# gb  
* master                cb046a0 Merge pull request #196 from JedMeister/tweak-apt
  remotes/origin/HEAD   -> origin/master
  remotes/origin/master cb046a0 Merge pull request #196 from JedMeister/tweak-apt

root@tkldev .../fab/common# git remote set-branches origin '*'

root@tkldev .../fab/common# git fetch origin 
remote: Enumerating objects: 7184, done.
remote: Counting objects: 100% (7182/7182), done.
remote: Compressing objects: 100% (2749/2749), done.
remote: Total 6848 (delta 3029), reused 6694 (delta 2883), pack-reused 0
Receiving objects: 100% (6848/6848), 4.31 MiB | 5.06 MiB/s, done.
Resolving deltas: 100% (3029/3029), completed with 132 local objects.
From https://github.com/turnkeylinux/common
 * [new branch]      15.x       -> origin/15.x
 * [new branch]      16.x       -> origin/16.x
 * [new branch]      17.x-dev   -> origin/17.x-dev
 * [new tag]         15.x       -> 15.x
 * [new tag]         11.3       -> 11.3
 * [new tag]         12.0       -> 12.0
 * [new tag]         12.1       -> 12.1
 * [new tag]         13.0       -> 13.0
 * [new tag]         14.0       -> 14.0
 * [new tag]         14.1       -> 14.1
 * [new tag]         14.2       -> 14.2
 * [new tag]         16.x       -> 16.x

root@tkldev .../fab/common# gb # git branch -av
* master                  cb046a0 Merge pull request #196 from JedMeister/tweak-apt
  remotes/origin/15.x     de4d093 Merge branch 'gh_releases'
  remotes/origin/16.x     cb046a0 Merge pull request #196 from JedMeister/tweak-apt
  remotes/origin/17.x-dev 4f38f1c Merge pull request #200 from OnGle/17.0-mysql-update
  remotes/origin/HEAD     -> origin/master
  remotes/origin/master   cb046a0 Merge pull request #196 from JedMeister/tweak-apt

[JedMeister]

@OnGle - What do you think? Should we default to full clones (to avoid this issue) or just document the required action to support pulling from alternate branches?

[OnGle]

Another option is to pass --no-single-branch I only just found this while beginning to post, but here's an explanation better than I could do myself: https://stackoverflow.com/a/23710503

[JedMeister]

Another option is to pass --no-single-branch

That's the go I reckon!