Closed
Description
Describe the bug
After creating an OSF sibling, the first datalad push --to osf succeeds, but the second one fails with [rejected] (fetch first)]. Inspecting the git repositories shows that the commit IDs are different in the OSF remote.
To Reproduce
Steps to reproduce the behavior:
$ datalad create example-osf
[INFO ] Creating a new annex repo at /home/pflueger/work/dl-test/example-osf
[INFO ] Scanning for unlocked files (this may take some time)
create(ok): /home/pflueger/work/dl-test/example-osf (dataset)
$ cd example-osf/
$ wget http://www.tldp.org/LDP/Bash-Beginners-Guide/Bash-Beginners-Guide.pdf
[…]
$ datalad save -m 'a pdf'
add(ok): Bash-Beginners-Guide.pdf (file)
save(ok): . (dataset)
action summary:
add (ok: 1)
save (ok: 1)
$ vim README.md
$ git add README.md
$ git commit -m 'thing'
[master e6db074] thing
1 file changed, 2 insertions(+)
create mode 100644 README.md
$ datalad create-sibling-osf --title example-osf
create-sibling-osf(ok): https://osf.io/hf624/
[INFO ] Configure additional publication dependency on "osf-storage"
configure-sibling(ok): . (sibling)
$ datalad push --to osf
copy(ok): Bash-Beginners-Guide.pdf (file) [to osf-storage...]
publish(ok): . (dataset) [refs/heads/master->osf:refs/heads/master [new branch]]
publish(ok): . (dataset) [refs/heads/git-annex->osf:refs/heads/git-annex [new branch]]
$ datalad push --to osf
[ERROR ] refs/heads/master->osf:refs/heads/master [rejected] (fetch first) [publish(/home/pflueger/work/dl-test/example-osf)]
publish(error): . (dataset) [refs/heads/master->osf:refs/heads/master [rejected] (fetch first)]
Potential hints to solve encountered errors:
1: Updates were rejected because the remote contains work that you do not have locally. This is usually caused by another repository pushing to the same ref. You may want to first integrate the remote changes (e.g., 'git pull ...') before pushing again. See the 'Note about fast-forwards' in 'git push --help' for details.
$ git log master
commit e6db074a5c196839be747a178739523bc00ab5c2 (HEAD -> master, osf/master, refs/osf/osf/master)
Author: Mika Pflüger <mika.pflueger@pik-potsdam.de>
Date: Fri May 7 17:25:58 2021 +0200
thing
commit 7cf3bc9bb4f86fb33f08b7f7a3d259a6ffe190eb
Author: Mika Pflüger <mika.pflueger@pik-potsdam.de>
Date: Fri May 7 17:25:29 2021 +0200
a pdf
commit 4909c54ee3c4288791e1fdf70ad0ef98f54c14ee
Author: Mika Pflüger <mika.pflueger@pik-potsdam.de>
Date: Fri May 7 17:25:16 2021 +0200
[DATALAD] new dataset
$ # compare with what we get in a fresh clone of the git repo
$ cd ..
$ git clone osf://hf624
Cloning into 'hf624'...
100%|█████████████████| 145/145 [00:00<00:00, 934kbytes/s]
Downloading repository archive
100%|█████████████████| 18.6k/18.6k [00:00<00:00, 5.51Mbytes/s]
Extracting repository archive
$ cd hf624/
Bash-Beginners-Guide.pdf@ README.md
$ git log
commit f6e3ce21a7d6a5f0d8d820a4f40873c9370a68ba (HEAD -> master, origin/master, origin/HEAD, refs/osf/origin/master)
Author: Mika Pflüger <mika.pflueger@pik-potsdam.de>
Date: Fri May 7 17:25:58 2021 +0200
thing
commit ba3846a3968517b4017b33f6a1f5ff573cbddbee
Author: Mika Pflüger <mika.pflueger@pik-potsdam.de>
Date: Fri May 7 17:25:29 2021 +0200
a pdf
commit 2246d22cc3758b612f75862e35ca304e4956cedb
Author: Mika Pflüger <mika.pflueger@pik-potsdam.de>
Date: Fri May 7 17:25:16 2021 +0200
[DATALAD] new dataset
As you can see, the ID of the first commit is 4909c54ee3c in the original datalad repo, and 2246d22cc375 in the clone obtained from OSF.
Desktop (please complete the following information):
- OS: Ubuntu 21-04
Datalad WTF
# WTF
## configuration <SENSITIVE, report disabled by configuration>
## credentials
- keyring:
- active_backends:
- SecretService Keyring
- kwallet DBusKeyring
- EncryptedKeyring with [PBKDF2] AES256.CFB v.1.0 at /home/pflueger/.local/share/python_keyring/crypted_pass.cfg
- PlaintextKeyring with no encyption v.1.0 at /home/pflueger/.local/share/python_keyring/keyring_pass.cfg
- config_file: /home/pflueger/.config/python_keyring/keyringrc.cfg
- data_root: /home/pflueger/.local/share/python_keyring
## datalad
- full_version: 0.14.3
- version: 0.14.3
## dataset
- id: 9f60be2e-3205-43be-9f1b-acecc95a1649
- metadata: <SENSITIVE, report disabled by configuration>
- path: /tmp/hf624
- repo: AnnexRepo
## dependencies
- annexremote: 1.4.3
- appdirs: 1.4.4
- boto: 2.49.0
- cmd:7z: 16.02
- cmd:annex: 8.20210223-1~ndall+1
- cmd:bundled-git: 2.24.0
- cmd:git: 2.24.0
- cmd:system-git: 2.30.2
- cmd:system-ssh: 8.4p1
- exifread: 2.3.2
- git: 3.1.12
- gitdb: 4.0.5
- humanize: 0.0.0
- iso8601: 0.1.13
- keyring: 22.2.0
- keyrings.alt: 4.0.2
- msgpack: 1.0.0
- mutagen: 1.45.1
- requests: 2.25.1
- wrapt: 1.12.1
## environment
- LANG: en_US.UTF-8
- LANGUAGE: en_US:
- LC_ADDRESS: de_DE.UTF-8
- LC_IDENTIFICATION: de_DE.UTF-8
- LC_MEASUREMENT: de_DE.UTF-8
- LC_MONETARY: de_DE.UTF-8
- LC_NAME: de_DE.UTF-8
- LC_NUMERIC: de_DE.UTF-8
- LC_PAPER: de_DE.UTF-8
- LC_TELEPHONE: de_DE.UTF-8
- LC_TIME: de_DE.UTF-8
- PATH: /home/pflueger/.pyenv/shims:/home/pflueger/.local/conda/condabin:/home/pflueger/.local/bin:/home/pflueger/go/bin:/home/pflueger/.pyenv/bin:/home/pflueger/.poetry/bin:/home/pflueger/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
## extensions
- osf:
- description: DataLad extension for OSF support
- entrypoints:
- datalad_osf.create_sibling_osf.CreateSiblingOSF:
- class: CreateSiblingOSF
- load_error: None
- module: datalad_osf.create_sibling_osf
- names:
- create-sibling-osf
- create_sibling_osf
- datalad_osf.credentials.OSFCredentials:
- class: OSFCredentials
- load_error: None
- module: datalad_osf.credentials
- names:
- osf-credentials
- osf_credentials
- load_error: None
- module: datalad_osf
- version: 0.2.3.1
## git-annex
- build flags:
- Assistant
- Webapp
- Pairing
- Inotify
- DBus
- DesktopNotify
- TorrentParser
- MagicMime
- Feeds
- Testsuite
- S3
- WebDAV
- dependency versions:
- aws-0.20
- bloomfilter-2.0.1.0
- cryptonite-0.25
- DAV-1.3.3
- feed-1.0.0.0
- ghc-8.4.4
- http-client-0.5.13.1
- persistent-sqlite-2.8.2
- torrent-10000.1.1
- uuid-1.3.13
- yesod-1.6.0
- key/value backends:
- SHA256E
- SHA256
- SHA512E
- SHA512
- SHA224E
- SHA224
- SHA384E
- SHA384
- SHA3_256E
- SHA3_256
- SHA3_512E
- SHA3_512
- SHA3_224E
- SHA3_224
- SHA3_384E
- SHA3_384
- SKEIN256E
- SKEIN256
- SKEIN512E
- SKEIN512
- BLAKE2B256E
- BLAKE2B256
- BLAKE2B512E
- BLAKE2B512
- BLAKE2B160E
- BLAKE2B160
- BLAKE2B224E
- BLAKE2B224
- BLAKE2B384E
- BLAKE2B384
- BLAKE2BP512E
- BLAKE2BP512
- BLAKE2S256E
- BLAKE2S256
- BLAKE2S160E
- BLAKE2S160
- BLAKE2S224E
- BLAKE2S224
- BLAKE2SP256E
- BLAKE2SP256
- BLAKE2SP224E
- BLAKE2SP224
- SHA1E
- SHA1
- MD5E
- MD5
- WORM
- URL
- X*
- operating system: linux x86_64
- remote types:
- git
- gcrypt
- p2p
- S3
- bup
- directory
- rsync
- web
- bittorrent
- webdav
- adb
- tahoe
- glacier
- ddar
- git-lfs
- httpalso
- borg
- hook
- external
- supported repository versions:
- 8
- upgrade supported from repository versions:
- 0
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- version: 8.20210223-1~ndall+1
## location
- path: /tmp/hf624
- type: dataset
## metadata_extractors
- annex (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.annex
- version: None
- audio (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.audio
- version: None
- datacite (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.datacite
- version: None
- datalad_core (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.datalad_core
- version: None
- datalad_rfc822 (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.datalad_rfc822
- version: None
- exif (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.exif
- version: None
- frictionless_datapackage (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.frictionless_datapackage
- version: None
- image (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: None
- module: datalad.metadata.extractors.image
- version: None
- xmp (datalad 0.14.3):
- distribution: datalad 0.14.3
- load_error: No module named 'libxmp' [xmp.py:<module>:20]
- module: datalad.metadata.extractors.xmp
## metadata_indexers
## python
- implementation: CPython
- version: 3.9.4
## system
- distribution: ubuntu/21.04/hirsute
- encoding:
- default: utf-8
- filesystem: utf-8
- locale.prefered: UTF-8
- max_path_length: 266
- name: Linux
- release: 5.11.0-16-generic
- type: posix
- version: #17-Ubuntu SMP Wed Apr 14 20:12:43 UTC 2021
Expected behavior
I expect that following the tutorial, I get a datalad repo with an OSF sibling which can be pushed multiple times.
Additional Info
I haven't looked into datalad-osf code myself yet, but I think that the inconsistency comes straight from git-remote-osf.