Description
Describe the bug
Files/folders from a Personal Shared Folder cannot be filtered because the entries are not considered with their full paths. I.e. the contents of sync_list create no match.
Consider the following example setup:
OneDrive Root --> Test 1 (actual folder)
OneDrive Root --> OneDrive - Constanze (shared folder from another account)
The sync_list is as follows (%20 entries are necessary because I cannot get rid of the url-encoded spaces, cf. #2562)
/OneDrive - Constanze/*
/OneDrive%20-%20Constanze/*
/Test 1/*
/Test%201/*
The relevant part of the log is:
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive JSON item 3 of 3 as part of JSON Item Batch 1 of 1
DEBUG: Raw JSON OneDrive Item (Batched Item): {"cTag":"aYzo5QkE2RjA3NjY4QkExRkY3ITIyMzg3LjI1OA","eTag":"aOUJBNkYwNzY2OEJBMUZGNyEyMjM4Ny4z","file":{"hashes":{"quickXorHash":"YZgDGcwAAAAAAAAABAAAAAAAAAA=","sha1Hash":"3DA541559918A808C2402BBA5012F6C60B27661C","sha256Hash":"F0E4C2F76C58916EC258F246851BEA091D14D4247A2FC3E18694461B1816E13B"},"mimeType":"text\/plain"},"fileSystemInfo":{"createdDateTime":"2024-05-31T21:32:04.763Z","lastModifiedDateTime":"2024-05-31T21:32:27Z"},"id":"9BA6F07668BA1FF7!22387","name":"test1doc.txt","parentReference":{"driveId":"9ba6f07668ba1ff7","driveType":"personal","id":"9BA6F07668BA1FF7!22385","name":"Test 1","path":"\/drive\/root:\/Test%201"},"size":4}
DEBUG: Attempting to calculate local filesystem path for 9ba6f07668ba1ff7 and 9BA6F07668BA1FF7!22385
DEBUG: JSON Item calculated full path is: Test 1/test1doc.txt
DEBUG: The item we are syncing is a file
DEBUG: skip_file item to check: /Test 1/test1doc.txt
DEBUG: skip_file evaluation for: /Test 1/test1doc.txt
DEBUG: Result: false
DEBUG: sync_list item to check: Test 1/test1doc.txt
DEBUG: Evaluation against 'sync_list' for this path: Test 1/test1doc.txt
DEBUG: [S]exclude = false
DEBUG: [S]exludeDirectMatch = false
DEBUG: [S]excludeMatched = false
DEBUG: Evaluation against 'sync_list' entry: /OneDrive - Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /OneDrive%20-%20Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /Test 1/*
DEBUG: Evaluation against 'sync_list' result: wildcard pattern match
DEBUG: Evaluation against 'sync_list' entry: /Test%201/*
DEBUG: [F]exclude = false
DEBUG: [F]exludeDirectMatch = false
DEBUG: [F]excludeMatched = false
DEBUG: Evaluation against 'sync_list' final result: included for sync
DEBUG: Flagging object as a file
DEBUG: OneDrive change is potentially a new local item
DEBUG: Path on local disk already exists
DEBUG: Local file time discrepancy detected: Test 1/test1doc.txt
DEBUG: This local file has a different modified time 2024-May-31 22:21:54Z (UTC) when compared to remote modified time 2024-May-31 21:32:27Z (UTC)
DEBUG: Local item has the same hash value as the item online - correcting the applicable file timestamp
DEBUG: The source of the incorrect timestamp was the local file - correcting timestamp locally due to --download-only
DEBUG: Calling setTimes() for this file: Test 1/test1doc.txt
DEBUG: The item to sync exists locally but is potentially not in the local database - otherwise this would be handled as changed item
DEBUG: File timestamps are equal, no further action required
DEBUG: Update/Insert local database with item details: Item("9ba6f07668ba1ff7", "9BA6F07668BA1FF7!22387", "test1doc.txt", "", file, "aOUJBNkYwNzY2OEJBMUZGNyEyMjM4Ny4z", "aYzo5QkE2RjA3NjY4QkExRkY3ITIyMzg3LjI1OA", 2024-May-31 21:32:27Z, "9BA6F07668BA1FF7!22385", "YZgDGcwAAAAAAAAABAAAAAAAAAA=", "", "", "", "", none, "Y", "4")
DEBUG: Batched JSON item processing time: 345 ms, 935 μs, and 1 hnsec
DEBUG: ------------------------------------------------------------------
DEBUG: Syncing this OneDrive Personal Shared Folder: OneDrive - Constanze
DEBUG: Why are we generating a /delta response:
DEBUG: singleDirectoryScope: false
DEBUG: nationalCloudDeployment: false
DEBUG: cleanupLocalFiles: true
DEBUG: CurlEngine getCurlInstance() called
DEBUG: CurlEngine curlEnginePool current size: 3
DEBUG: CurlEngine was in a valid state - returning existing CurlEngine instance
DEBUG: CurlEngine instance ID: HRAP4XTgEzLNoBfl
DEBUG: Read token from appConfig
DEBUG: Authorised State: true
DEBUG: Request URL = https://graph.microsoft.com/v1.0/me/drive/root:/OneDrive%20-%20Constanze:/?select=id,name,eTag,cTag,deleted,file,folder,root,fileSystemInfo,remoteItem,parentReference,size
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:54 GMT", "client-request-id":"6a37e3e3-ba9a-41a3-943d-f065d86740a5", "request-id":"6a37e3e3-ba9a-41a3-943d-f065d86740a5", "cache-control":"private", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200 (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Downgrading all children for this searchItem.driveId (f3028bd758552faf) and searchItem.id (F3028BD758552FAF!4690) to an out-of-sync state
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/f3028bd758552faf/items/F3028BD758552FAF!4690?select=id,name,eTag,cTag,deleted,file,folder,root,fileSystemInfo,remoteItem,parentReference,size
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:54 GMT", "client-request-id":"afe5320a-8d04-4deb-981d-7f93723c60d6", "request-id":"afe5320a-8d04-4deb-981d-7f93723c60d6", "cache-control":"private", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200 (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Generating a /delta response from the OneDrive API for Drive ID: f3028bd758552faf
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/f3028bd758552faf/root
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:55 GMT", "client-request-id":"945efaff-e09d-4d67-9cb5-c17e15e442ba", "request-id":"945efaff-e09d-4d67-9cb5-c17e15e442ba", "cache-control":"no-store", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200 (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Adding OneDrive root details for processing
DEBUG: Adding OneDrive folder details for processing
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/f3028bd758552faf/items/F3028BD758552FAF!4690/children?select=id,name,eTag,cTag,deleted,file,folder,root,fileSystemInfo,remoteItem,parentReference,size
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:56 GMT", "client-request-id":"e6db94bd-78d3-4a46-86bb-065fec836c7b", "request-id":"e6db94bd-78d3-4a46-86bb-065fec836c7b", "cache-control":"no-store", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200 (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Adding 13 OneDrive items for processing from the OneDrive 'root' folder
DEBUG: sync_list item to check: Basteln
DEBUG: Evaluation against 'sync_list' for this path: Basteln
DEBUG: [S]exclude = false
DEBUG: [S]exludeDirectMatch = false
DEBUG: [S]excludeMatched = false
DEBUG: Evaluation against 'sync_list' entry: /OneDrive - Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /OneDrive%20-%20Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /Test 1/*
DEBUG: Evaluation against 'sync_list' entry: /Test%201/*
DEBUG: [F]exclude = false
DEBUG: [F]exludeDirectMatch = false
DEBUG: [F]excludeMatched = false
DEBUG: Evaluation against 'sync_list' final result: EXCLUDED
DEBUG: Skipping item - excluded by sync_list config: Basteln
So unfortunately the folder Basteln from OneDrive - Constanze is not synced while a test file within Test 1 folder is synced properly. The expected behaviour would be to instead of
DEBUG: Evaluation against 'sync_list' for this path: Basteln
have:
DEBUG: Evaluation against 'sync_list' for this path: OneDrive - Constanze/Basteln
FYI: I was able to solve #2620 by upgrading smbclient to 4.19.5 (manual compile in Ubuntu). Unfortunately, I cannot write the information to #2620 because it is already closed.
Operating System Details
Linux atom 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.4 LTS
Release: 22.04
Codename: jammy
Client Installation Method
From Source
OneDrive Account Type
Personal
What is your OneDrive Application Version
onedrive v2.5.0-rc2-25-g71bce90
What is your OneDrive Application Configuration
onedrive version = onedrive v2.5.0-rc2-25-g71bce90
Config path = /home/robert/.config/onedrive
Config file found in config path = false
Config option 'drive_id' =
Config option 'sync_dir' = ~/OneDrive
Config option 'enable_logging' = false
Config option 'log_dir' = /var/log/onedrive
Config option 'disable_notifications' = false
Config option 'skip_dir' =
Config option 'skip_dir_strict_match' = false
Config option 'skip_file' = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles' = false
Config option 'skip_symlinks' = false
Config option 'monitor_interval' = 300
Config option 'monitor_log_frequency' = 12
Config option 'monitor_fullscan_frequency' = 12
Config option 'read_only_auth_scope' = false
Config option 'dry_run' = false
Config option 'upload_only' = false
Config option 'download_only' = false
Config option 'local_first' = false
Config option 'check_nosync' = false
Config option 'check_nomount' = false
Config option 'resync' = false
Config option 'resync_auth' = false
Config option 'cleanup_local_files' = false
Config option 'classify_as_big_delete' = 1000
Config option 'disable_upload_validation' = false
Config option 'disable_download_validation' = false
Config option 'bypass_data_preservation' = false
Config option 'no_remote_delete' = false
Config option 'remove_source_files' = false
Config option 'sync_dir_permissions' = 700
Config option 'sync_file_permissions' = 600
Config option 'space_reservation' = 52428800
Config option 'application_id' = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint' =
Config option 'azure_tenant_id' =
Config option 'user_agent' = ISV|abraunegg|OneDrive Client for Linux/v2.5.0-rc2-25-g71bce90
Config option 'force_http_11' = false
Config option 'debug_https' = false
Config option 'rate_limit' = 0
Config option 'operation_timeout' = 3600
Config option 'dns_timeout' = 60
Config option 'connect_timeout' = 10
Config option 'data_timeout' = 60
Config option 'ip_protocol_version' = 0
Config option 'threads' = 8
Selective sync 'sync_list' configured = false
Config option 'sync_business_shared_items' = false
Config option 'webhook_enabled' = false
What is your 'curl' version
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.17
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd
Where is your 'sync_dir' located
Network
What are all your system 'mount points'
//hyperv/3 on /media/3 type cifs (rw,relatime,vers=3.1.1,cache=strict,username=Administrator,uid=1000,noforceuid,gid=100,noforcegid,addr=192.168.3.11,file_mode=0777,dir_mode=0777,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)
What are all your local file system partition types
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0 squashfs 4.0 0 100% /snap/bare/5
loop1 squashfs 4.0 0 100% /snap/canonical-livepatch/264
loop2 squashfs 4.0 0 100% /snap/canonical-livepatch/278
loop3 squashfs 4.0 0 100% /snap/core/16574
loop4 squashfs 4.0 0 100% /snap/core/16928
loop5 squashfs 4.0 0 100% /snap/core18/2812
loop6 squashfs 4.0 0 100% /snap/core18/2823
loop7 squashfs 4.0 0 100% /snap/core20/2264
loop8 squashfs 4.0 0 100% /snap/core20/2318
loop9 squashfs 4.0 0 100% /snap/core22/1122
loop10 squashfs 4.0 0 100% /snap/core22/1380
loop11 squashfs 4.0 0 100% /snap/cups/1044
loop12 squashfs 4.0 0 100% /snap/cups/1047
loop14 squashfs 4.0 0 100% /snap/firefox/4259
loop15 squashfs 4.0 0 100% /snap/gnome-3-26-1604/104
loop16 squashfs 4.0 0 100% /snap/gnome-3-26-1604/111
loop17 squashfs 4.0 0 100% /snap/gnome-3-28-1804/194
loop18 squashfs 4.0 0 100% /snap/gnome-3-28-1804/198
loop19 squashfs 4.0 0 100% /snap/gnome-3-34-1804/90
loop20 squashfs 4.0 0 100% /snap/gnome-3-34-1804/93
loop21 squashfs 4.0 0 100% /snap/gnome-3-38-2004/140
loop22 squashfs 4.0 0 100% /snap/gnome-3-38-2004/143
loop23 squashfs 4.0 0 100% /snap/gnome-42-2204/172
loop24 squashfs 4.0 0 100% /snap/gnome-42-2204/176
loop25 squashfs 4.0 0 100% /snap/gnome-system-monitor/184
loop26 squashfs 4.0 0 100% /snap/gnome-system-monitor/186
loop27 squashfs 4.0 0 100% /snap/gtk-common-themes/1534
loop28 squashfs 4.0 0 100% /snap/gtk-common-themes/1535
loop29 squashfs 4.0 0 100% /snap/snap-store/1113
loop30 squashfs 4.0 0 100% /snap/snap-store/959
loop31 squashfs 4.0 0 100% /snap/snapd-desktop-integration/157
loop32 squashfs 4.0 0 100% /snap/snapd-desktop-integration/83
loop33 0 100% /snap/firefox/4336
sda
├─sda1 vfat FAT32 62EB-8E64 503,3M 1% /boot/efi
├─sda2
└─sda5 LVM2_member LVM2 001 X34xd2-HOp4-sAHk-pudA-CzVG-rENF-FOtrO2
├─ubuntu--vg-root ext4 1.0 43f5fc57-8370-4315-b5ba-95912f21e113 20,8G 74% /var/snap/firefox/common/host-hunspell
│ /media/robert_home
│ /
└─ubuntu--vg-swap_1 swap 1 ad32bbec-507e-44b0-bb5f-1df3ee647a2a [SWAP]
How do you use 'onedrive'
cf. main bug description above
Steps to reproduce the behaviour
./onedrive --confdir='/home/robert/.config/onedrive/accounts/robert@guitaronline.de' --sync --verbose --verbose --resync --resync-auth > debug_output.log 2>&1
Complete Verbose Log Output
Can provide via mail in case needed.
Screenshots
No response
Other Log Information or Details
No response
Additional context
No response