Skip to content

Bug: Cannot select/filter within Personal Shared Folders #2731

Closed
@robertschulze

Description

Describe the bug

Files/folders from a Personal Shared Folder cannot be filtered because the entries are not considered with their full paths. I.e. the contents of sync_list create no match.

Consider the following example setup:
OneDrive Root --> Test 1 (actual folder)
OneDrive Root --> OneDrive - Constanze (shared folder from another account)

The sync_list is as follows (%20 entries are necessary because I cannot get rid of the url-encoded spaces, cf. #2562)

/OneDrive - Constanze/*                                                         
/OneDrive%20-%20Constanze/*                                                     
/Test 1/*                                                                       
/Test%201/*                                                                     

The relevant part of the log is:

DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive JSON item 3 of 3 as part of JSON Item Batch 1 of 1
DEBUG: Raw JSON OneDrive Item (Batched Item): {"cTag":"aYzo5QkE2RjA3NjY4QkExRkY3ITIyMzg3LjI1OA","eTag":"aOUJBNkYwNzY2OEJBMUZGNyEyMjM4Ny4z","file":{"hashes":{"quickXorHash":"YZgDGcwAAAAAAAAABAAAAAAAAAA=","sha1Hash":"3DA541559918A808C2402BBA5012F6C60B27661C","sha256Hash":"F0E4C2F76C58916EC258F246851BEA091D14D4247A2FC3E18694461B1816E13B"},"mimeType":"text\/plain"},"fileSystemInfo":{"createdDateTime":"2024-05-31T21:32:04.763Z","lastModifiedDateTime":"2024-05-31T21:32:27Z"},"id":"9BA6F07668BA1FF7!22387","name":"test1doc.txt","parentReference":{"driveId":"9ba6f07668ba1ff7","driveType":"personal","id":"9BA6F07668BA1FF7!22385","name":"Test 1","path":"\/drive\/root:\/Test%201"},"size":4}
DEBUG: Attempting to calculate local filesystem path for 9ba6f07668ba1ff7 and 9BA6F07668BA1FF7!22385
DEBUG: JSON Item calculated full path is: Test 1/test1doc.txt
DEBUG: The item we are syncing is a file
DEBUG: skip_file item to check: /Test 1/test1doc.txt
DEBUG: skip_file evaluation for: /Test 1/test1doc.txt
DEBUG: Result: false
DEBUG: sync_list item to check: Test 1/test1doc.txt
DEBUG: Evaluation against 'sync_list' for this path: Test 1/test1doc.txt
DEBUG: [S]exclude           = false
DEBUG: [S]exludeDirectMatch = false
DEBUG: [S]excludeMatched    = false
DEBUG: Evaluation against 'sync_list' entry: /OneDrive - Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /OneDrive%20-%20Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /Test 1/*
DEBUG: Evaluation against 'sync_list' result: wildcard pattern match
DEBUG: Evaluation against 'sync_list' entry: /Test%201/*
DEBUG: [F]exclude           = false
DEBUG: [F]exludeDirectMatch = false
DEBUG: [F]excludeMatched    = false
DEBUG: Evaluation against 'sync_list' final result: included for sync
DEBUG: Flagging object as a file
DEBUG: OneDrive change is potentially a new local item
DEBUG: Path on local disk already exists
DEBUG: Local file time discrepancy detected: Test 1/test1doc.txt
DEBUG: This local file has a different modified time 2024-May-31 22:21:54Z (UTC) when compared to remote modified time 2024-May-31 21:32:27Z (UTC)
DEBUG: Local item has the same hash value as the item online - correcting the applicable file timestamp
DEBUG: The source of the incorrect timestamp was the local file - correcting timestamp locally due to --download-only
DEBUG: Calling setTimes() for this file: Test 1/test1doc.txt
DEBUG: The item to sync exists locally but is potentially not in the local database - otherwise this would be handled as changed item
DEBUG: File timestamps are equal, no further action required
DEBUG: Update/Insert local database with item details: Item("9ba6f07668ba1ff7", "9BA6F07668BA1FF7!22387", "test1doc.txt", "", file, "aOUJBNkYwNzY2OEJBMUZGNyEyMjM4Ny4z", "aYzo5QkE2RjA3NjY4QkExRkY3ITIyMzg3LjI1OA", 2024-May-31 21:32:27Z, "9BA6F07668BA1FF7!22385", "YZgDGcwAAAAAAAAABAAAAAAAAAA=", "", "", "", "", none, "Y", "4")
DEBUG: Batched JSON item processing time: 345 ms, 935 μs, and 1 hnsec
DEBUG: ------------------------------------------------------------------
DEBUG: Syncing this OneDrive Personal Shared Folder: OneDrive - Constanze
DEBUG: Why are we generating a /delta response:
DEBUG:  singleDirectoryScope:    false
DEBUG:  nationalCloudDeployment: false
DEBUG:  cleanupLocalFiles:       true
DEBUG: CurlEngine getCurlInstance() called
DEBUG: CurlEngine curlEnginePool current size: 3
DEBUG: CurlEngine was in a valid state - returning existing CurlEngine instance
DEBUG: CurlEngine instance ID: HRAP4XTgEzLNoBfl
DEBUG: Read token from appConfig
DEBUG: Authorised State: true
DEBUG: Request URL = https://graph.microsoft.com/v1.0/me/drive/root:/OneDrive%20-%20Constanze:/?select=id,name,eTag,cTag,deleted,file,folder,root,fileSystemInfo,remoteItem,parentReference,size
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:54 GMT", "client-request-id":"6a37e3e3-ba9a-41a3-943d-f065d86740a5", "request-id":"6a37e3e3-ba9a-41a3-943d-f065d86740a5", "cache-control":"private", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200  (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Downgrading all children for this searchItem.driveId (f3028bd758552faf) and searchItem.id (F3028BD758552FAF!4690) to an out-of-sync state
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/f3028bd758552faf/items/F3028BD758552FAF!4690?select=id,name,eTag,cTag,deleted,file,folder,root,fileSystemInfo,remoteItem,parentReference,size
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:54 GMT", "client-request-id":"afe5320a-8d04-4deb-981d-7f93723c60d6", "request-id":"afe5320a-8d04-4deb-981d-7f93723c60d6", "cache-control":"private", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200  (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Generating a /delta response from the OneDrive API for Drive ID: f3028bd758552faf
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/f3028bd758552faf/root
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:55 GMT", "client-request-id":"945efaff-e09d-4d67-9cb5-c17e15e442ba", "request-id":"945efaff-e09d-4d67-9cb5-c17e15e442ba", "cache-control":"no-store", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200  (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Adding OneDrive root details for processing
DEBUG: Adding OneDrive folder details for processing
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/f3028bd758552faf/items/F3028BD758552FAF!4690/children?select=id,name,eTag,cTag,deleted,file,folder,root,fileSystemInfo,remoteItem,parentReference,size
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jun-01 01:21:53.3723092
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"Germany West Central\",\"Slice\":\"E\",\"Ring\":\"4\",\"ScaleUnit\":\"001\",\"RoleInstance\":\"FR2PEPF000001E9\"}}", "odata-version":"4.0", "date":"Fri, 31 May 2024 22:21:56 GMT", "client-request-id":"e6db94bd-78d3-4a46-86bb-065fec836c7b", "request-id":"e6db94bd-78d3-4a46-86bb-065fec836c7b", "cache-control":"no-store", "content-type":"application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"]
DEBUG: HTTP Status Line: 200  (2.0)
DEBUG: CurlEngine cleanup() called on instance id: HRAP4XTgEzLNoBfl
DEBUG: Adding 13 OneDrive items for processing from the OneDrive 'root' folder
DEBUG: sync_list item to check: Basteln
DEBUG: Evaluation against 'sync_list' for this path: Basteln
DEBUG: [S]exclude           = false
DEBUG: [S]exludeDirectMatch = false
DEBUG: [S]excludeMatched    = false
DEBUG: Evaluation against 'sync_list' entry: /OneDrive - Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /OneDrive%20-%20Constanze/*
DEBUG: Evaluation against 'sync_list' entry: /Test 1/*
DEBUG: Evaluation against 'sync_list' entry: /Test%201/*
DEBUG: [F]exclude           = false
DEBUG: [F]exludeDirectMatch = false
DEBUG: [F]excludeMatched    = false
DEBUG: Evaluation against 'sync_list' final result: EXCLUDED
DEBUG: Skipping item - excluded by sync_list config: Basteln

So unfortunately the folder Basteln from OneDrive - Constanze is not synced while a test file within Test 1 folder is synced properly. The expected behaviour would be to instead of

DEBUG: Evaluation against 'sync_list' for this path: Basteln

have:

DEBUG: Evaluation against 'sync_list' for this path: OneDrive - Constanze/Basteln

FYI: I was able to solve #2620 by upgrading smbclient to 4.19.5 (manual compile in Ubuntu). Unfortunately, I cannot write the information to #2620 because it is already closed.

Operating System Details

Linux atom 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.4 LTS
Release:        22.04
Codename:       jammy

Client Installation Method

From Source

OneDrive Account Type

Personal

What is your OneDrive Application Version

onedrive v2.5.0-rc2-25-g71bce90

What is your OneDrive Application Configuration

onedrive version                             = onedrive v2.5.0-rc2-25-g71bce90
Config path                                  = /home/robert/.config/onedrive
Config file found in config path             = false
Config option 'drive_id'                     = 
Config option 'sync_dir'                     = ~/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.0-rc2-25-g71bce90
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 0
Config option 'threads'                      = 8

Selective sync 'sync_list' configured        = false

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.17
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Network

What are all your system 'mount points'

//hyperv/3 on /media/3 type cifs (rw,relatime,vers=3.1.1,cache=strict,username=Administrator,uid=1000,noforceuid,gid=100,noforcegid,addr=192.168.3.11,file_mode=0777,dir_mode=0777,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)

What are all your local file system partition types

NAME                  FSTYPE      FSVER    LABEL UUID                                   FSAVAIL FSUSE% MOUNTPOINTS
loop0                 squashfs    4.0                                                         0   100% /snap/bare/5
loop1                 squashfs    4.0                                                         0   100% /snap/canonical-livepatch/264
loop2                 squashfs    4.0                                                         0   100% /snap/canonical-livepatch/278
loop3                 squashfs    4.0                                                         0   100% /snap/core/16574
loop4                 squashfs    4.0                                                         0   100% /snap/core/16928
loop5                 squashfs    4.0                                                         0   100% /snap/core18/2812
loop6                 squashfs    4.0                                                         0   100% /snap/core18/2823
loop7                 squashfs    4.0                                                         0   100% /snap/core20/2264
loop8                 squashfs    4.0                                                         0   100% /snap/core20/2318
loop9                 squashfs    4.0                                                         0   100% /snap/core22/1122
loop10                squashfs    4.0                                                         0   100% /snap/core22/1380
loop11                squashfs    4.0                                                         0   100% /snap/cups/1044
loop12                squashfs    4.0                                                         0   100% /snap/cups/1047
loop14                squashfs    4.0                                                         0   100% /snap/firefox/4259
loop15                squashfs    4.0                                                         0   100% /snap/gnome-3-26-1604/104
loop16                squashfs    4.0                                                         0   100% /snap/gnome-3-26-1604/111
loop17                squashfs    4.0                                                         0   100% /snap/gnome-3-28-1804/194
loop18                squashfs    4.0                                                         0   100% /snap/gnome-3-28-1804/198
loop19                squashfs    4.0                                                         0   100% /snap/gnome-3-34-1804/90
loop20                squashfs    4.0                                                         0   100% /snap/gnome-3-34-1804/93
loop21                squashfs    4.0                                                         0   100% /snap/gnome-3-38-2004/140
loop22                squashfs    4.0                                                         0   100% /snap/gnome-3-38-2004/143
loop23                squashfs    4.0                                                         0   100% /snap/gnome-42-2204/172
loop24                squashfs    4.0                                                         0   100% /snap/gnome-42-2204/176
loop25                squashfs    4.0                                                         0   100% /snap/gnome-system-monitor/184
loop26                squashfs    4.0                                                         0   100% /snap/gnome-system-monitor/186
loop27                squashfs    4.0                                                         0   100% /snap/gtk-common-themes/1534
loop28                squashfs    4.0                                                         0   100% /snap/gtk-common-themes/1535
loop29                squashfs    4.0                                                         0   100% /snap/snap-store/1113
loop30                squashfs    4.0                                                         0   100% /snap/snap-store/959
loop31                squashfs    4.0                                                         0   100% /snap/snapd-desktop-integration/157
loop32                squashfs    4.0                                                         0   100% /snap/snapd-desktop-integration/83
loop33                                                                                        0   100% /snap/firefox/4336
sda                                                                                                    
├─sda1                vfat        FAT32          62EB-8E64                               503,3M     1% /boot/efi
├─sda2                                                                                                 
└─sda5                LVM2_member LVM2 001       X34xd2-HOp4-sAHk-pudA-CzVG-rENF-FOtrO2                
  ├─ubuntu--vg-root   ext4        1.0            43f5fc57-8370-4315-b5ba-95912f21e113     20,8G    74% /var/snap/firefox/common/host-hunspell
  │                                                                                                    /media/robert_home
  │                                                                                                    /
  └─ubuntu--vg-swap_1 swap        1              ad32bbec-507e-44b0-bb5f-1df3ee647a2a                  [SWAP]

How do you use 'onedrive'

cf. main bug description above

Steps to reproduce the behaviour

./onedrive --confdir='/home/robert/.config/onedrive/accounts/robert@guitaronline.de' --sync --verbose --verbose --resync --resync-auth > debug_output.log 2>&1

Complete Verbose Log Output

Can provide via mail in case needed.

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't workingFixed

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions