[Bug]: Upload into group folder causes a massive delay on move operation #47856
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
We use an S3 object store as primary storage for our Nextcloud cluster. We have recognized that uploads into a group folder cause a massive delay on the final file’s move operation, when the chunks are being assembled and the file gets created on the actual upload target:
Uploads of a 6 GB file cause a delay of 2.14 min and a 170 GB file a delay of 33 min.
When the upload appears outside of a group folder, the move operation does not cause any noticeable delay. During a normal upload the command sequence ends with a CompleteMultipartUpload. In case of group folder target, a lot of additional CopyObjectPart commands are being created and sent to the object storage. It depends on the file size, and the number of chunks, when the sequence ends.
I have attached a trace (refer to: Additional info) on a test bucket on our MinIO-S3-storage, which shows, what is actually happening. It looks like a copy-delete sequence initiated by the Nextcloud. Maybe the server thinks, that source and target are located on different storage systems.
Steps to reproduce
- Create a group folder.
- Upload a large file into this group folder.
- Wait until the move operation ends.
Expected behavior
Uploads into a group folder should not cause any noticeable delay on the final move operation.
Nextcloud Server version
28
Operating system
None
PHP engine version
None
Web server
None
Database engine version
None
Is this bug present after an update or on a fresh install?
None
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"memcache.local": "\\OC\\Memcache\\APCu",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"nextcloud-s3-t.mycompany.de"
],
"proxy": "proxy.mycompany.de:3128",
"proxyexclude": [
"127.0.0.0\/8",
"localhost",
".mycompany.de"
],
"objectstore": {
"class": "\\OC\\Files\\ObjectStore\\S3",
"arguments": {
"bucket": "ed-nxtcld-test",
"verify_bucket_exists": false,
"key": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"hostname": "nextcloud-minio-srv1-t.lx.mycompany.de",
"port": 9000,
"use_ssl": false,
"use_path_style": true,
"proxy": false
}
},
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "pgsql",
"version": "29.0.6.1",
"overwrite.cli.url": "https:\/\/nextcloud-s3-t.mycompany.de",
"overwriteprotocol": "https",
"overwritehost": "nextcloud-s3-t.mycompany.de",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"remember_login_cookie_lifetime": 1296000,
"session_lifetime": 86400,
"session_keepalive": true,
"auto_logout": true,
"mail_smtpmode": "sendmail",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": 25,
"mail_sendmailmode": "smtp",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"default_locale": "de_DE",
"default_phone_region": "DE",
"filelocking.enabled": true,
"htaccess.RewriteBase": "\/",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 6379,
"user": "default"
},
"theme": "",
"loglevel": 2,
"maintenance": false,
"maintenance_window_start": 1,
"ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
"updater.release.channel": "stable"
}
}List of activated Apps
Enabled:
- activity: 2.21.1
- circles: 29.0.0-dev
- cloud_federation_api: 1.12.0
- comments: 1.19.0
- contactsinteraction: 1.10.0
- dashboard: 7.9.0
- dav: 1.30.1
- federatedfilesharing: 1.19.0
- federation: 1.19.0
- files: 2.1.1
- files_downloadlimit: 2.0.0
- files_pdfviewer: 2.10.0
- files_reminders: 1.2.0
- files_sharing: 1.21.0
- files_trashbin: 1.19.0
- files_versions: 1.22.0
- firstrunwizard: 2.18.0
- groupfolders: 17.0.3
- logreader: 2.14.0
- lookup_server_connector: 1.17.0
- nextcloud_announcements: 1.18.0
- notifications: 2.17.0
- oauth2: 1.17.0
- password_policy: 1.19.0
- photos: 2.5.0
- privacy: 1.13.0
- provisioning_api: 1.19.0
- recommendations: 2.1.0
- related_resources: 1.4.0
- serverinfo: 1.19.0
- settings: 1.12.0
- sharebymail: 1.19.0
- support: 1.12.0
- survey_client: 1.17.0
- systemtags: 1.19.0
- text: 3.10.1
- theming: 2.4.0
- twofactor_backupcodes: 1.18.0
- updatenotification: 1.19.1
- user_status: 1.9.0
- viewer: 2.3.0
- weather_status: 1.9.0
- workflowengine: 2.11.0
Disabled:
- admin_audit: 1.19.0
- bruteforcesettings: 2.9.0
- encryption: 2.17.0
- files_external: 1.21.0
- suspicious_login: 7.0.0
- twofactor_totp: 11.0.0-dev
- user_ldap: 1.20.0Nextcloud Signing status
No errors have been found.Nextcloud Logs
No response
Activity