Nextcloud-Client creating conflicts when it should not

[Bockeman]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Expected behaviour

Conflicts should only be created when more than one client uploads to the server the same file with different contents.

Actual behaviour

I have several Nextcoud-client 3.0.1 all running on Win10 machines.
One of the sync folders is /AppData/Roaming/Mozilla; this is an "active" folder tree with several files changing continuously. But I have only one Mozilla Firefox browser open.

I am getting several conflicts generated every hour or so.

Steps to reproduce

1. Add a sync folder pointing at a directory where the contents are changing continuously.
2. Wait for conflicts!

Client configuration

Client version: 3.0.1
Operating system: Win10
OS language: English

Server configuration

Nextcloud version: 19.0.3
Storage backend (external storage): No external storage. Files on server are NFS mounted (actually gluster).

Logs

The following are generated from scripts running on a linux server with the Win10 machine mounted under
/mnt/veriulan/d/Data/BobW
and show 4 separate occassions (with dates shown) when conflicts were generated:

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 254622 2020-09-23 16:51 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 165141).jsonlz4
-rwxr-xr-x 1 bobw warren 254616 2020-09-23 16:51 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

2020-09-23 17:01:43
2020-09-23 17:01:55

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:20 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 172056).sqlite
-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:29 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

2020-09-23 17:29:59
2020-09-23 17:30:04

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:36 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 173605).sqlite
-rwxr-xr-x 1 bobw warren 114688 2020-09-23 17:56 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 175858).sqlite
-rwxr-xr-x 1 bobw warren 114688 2020-09-23 17:56 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

2020-09-23 19:39:10
2020-09-23 19:39:16

Compare sizes and dates

-rwxr-xr-x 1 bobw warren 261050 2020-09-23 20:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201018).baklz4
-rwxr-xr-x 1 bobw warren 264483 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.baklz4

-rwxr-xr-x 1 bobw warren 261112 2020-09-23 20:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201033).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 262513 2020-09-23 20:18 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201853).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 262061 2020-09-23 20:22 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 202248).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 264463 2020-09-23 20:38 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 203812).baklz4
-rwxr-xr-x 1 bobw warren 264483 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.baklz4

-rwxr-xr-x 1 bobw warren 264409 2020-09-23 20:38 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 203834).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4

-rwxr-xr-x 1 bobw warren 2523136 2020-09-23 20:08 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 200812).sqlite
-rwxr-xr-x 1 bobw warren 2531328 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite

2020-09-23 20:46:07
2020-09-23 20:46:13

[edmael]

Same is happening here with Win10, Nextcloud Desktop 3.0.1 and Nextcloud Server 13.0.5.

[codewichtel]

Same happening in Win10 , Nextcloud Desktop 3.0.1 and NC Server 16 17 18 19

[Bockeman]

Another example conflict. This conflict should not happen. The files changing on the client are due to activity. The only files changing on the server are the nextcloud-client uploads to the server.

As a separate issue, "Click for details" has no effect.
#1195

A debug log is here
nextcloud_200929_1637.log

[rasos]

After several conflicted copy incidents we have been observing randomly timestamp changes on some files on the storage backend. By appointing incrond on some frequently affected files (including files which are not of interest to nextcloud syncing like .htaccess) to report any changes to syslog we could exclude that the server itself is touching any files. Our storage backend (Hetzner Storage Box) is being mounted via sshfs (which turned out more performant and more reliable than Samba/CIFS that had problems with directory names containing spaces at the end). We also mount it read-only (!) on a second server to run backups every second day - which was roughly the frequency of some changed file timestamps. After pausing the backup process we have not seen any conflicted files anymore. When reading a file with the command less on the nextcloud, once we could observe, that this also changed the timestamp, but this behavior was not yet reproducible.

So as the problem of @Bockeman occurs also with files that are NFS mounted to gluster we should be looking more closely at the reliability of storage backends.

[pandiloko]

I'm having this problem with Android Client and Docker server installation (20.0.4). Sorry, I don't understand the nature of the problem but is it not possible to obtain a MD5 sum in case of conflict to let the user know that the changes are related to date (access, creation, whatever) and not content? Even a option to ignore date changes would be fine.

[rojaro]

I am having this issue with the Windows 10 laptop of my wife quite often (at least a dozen times in within the last 12 months):

• Client and server machines use the same NTP server.
• Windows client version is 3.1.1
• Server version is 20.0.5
• There is always a newer and an older file.
• Usually the newer file is on the client, rarely on the server.
• The difference between the timestamps is usually just a few minutes, but often also a day or two.
• More often than not, the changes have been made a few months in the past.
• All files have been uploaded to the server by the very same client and user (no other clients/users have access).
• Storage on the server resides on vanilla Ext4 filesystem which is exclusive to Nextcloud (No other services are using the storage space)

Conflict resolution is also really annoying as i can only resolve one file at a time:

• If i choose to keep a local or remote file the client starts synchronizing which takes some time (~ 10 seconds).
• If i try to resolve another file while the synchronization is still running, the client will reproducably crash and i have to manually restart it.

Once i've finished manually resolving the conflicts everything seems to be fine for some time until my wife calls me again because the conflicts are back.

No problems with our Linux and Android clients ...

[rasos]

We could nail down the problem with an external storage that was changing the wrong timestamp. Instead of atime (when a backup was reading files) it always changed mtime and ctime. Check it with the stat command on any file and see if that is changed correctly, even if you just read. We've built now our own storage system and are just migrating data.

[er-vin]

We could nail down the problem with an external storage that was changing the wrong timestamp. Instead of atime (when a backup was reading files) it always changed mtime and ctime. Check it with the stat command on any file and see if that is changed correctly, even if you just read. We've built now our own storage system and are just migrating data.

I advise everyone here to check this kind of things. Indeed with an external storage behaving that way there's nothing else the client can do but report conflicts.

[github-actions]

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

[Bockeman]

I'm now on Nextcloud Client version 3.1.3 and I am still getting conflicts.

The general case appears to be when client files change during a Nextcloud sync, with one client and (obviusly) one server. In my opinion, if there is only one client, there should never be any conflicts under any circumstances, whether files change during the sync process, or there are network drop-outs. This should be all managed by the local and server database files with appropriate checksums to cover potential network issues.

My experience continues to be that Nextcloud Client is creating conflicts when it should not.
(Though, in fairness, such conflicts have been occurring less frequently).

[ikogan]

I'm even seeing this issue when using the built in markdown editor in Nextcloud. I did stat on a file:

  File: CC Transition Plan.md
  Size: 6906            Blocks: 29         IO Block: 7168   regular file
Device: 4ah/74d Inode: 329327      Links: 1
Access: (0700/-rwx------)  Uid: (1234/   kogan)   Gid: (1234/   kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:39:43.303202466 -0500
Change: 2021-02-25 14:39:43.303202466 -0500
 Birth: 

Then I save the file in the nextcloud gui and run stat again:

  File: CC Transition Plan.md
  Size: 6905            Blocks: 29         IO Block: 7168   regular file
Device: 4ah/74d Inode: 329327      Links: 1
Access: (0700/-rwx------)  Uid: (1234/   kogan)   Gid: (1234/   kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:40:00.383787486 -0500
Change: 2021-02-25 14:40:00.383787486 -0500
 Birth: -

A few seconds later, the GUI will tell me the file was edited "outside of the editor". I then stat the file again:

  File: CC Transition Plan.md
  Size: 6905            Blocks: 29         IO Block: 7168   regular file
Device: 4ah/74d Inode: 329327      Links: 1
Access: (0700/-rwx------)  Uid: (1234/   kogan)   Gid: (1234/   kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:40:00.383787486 -0500
Change: 2021-02-25 14:40:00.383787486 -0500
 Birth: -

When I looked down at my local clock, it read 14:40 as well. I don't know the exact number of milliseconds but this does not seem to be working as intended. These files are indeed on an "External SSH" share.