[Bug]: Cronjob went OOM once a day, I suspect GenerateMetadataJob. #42793
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
Hi,
I've discovered that my nextcloud cronjob that runs every 15 minutes crashes ones per day 16:45
I'm running the cron with 1G memory limit to evoid the oom killer kick in:
php8.1 -d memory_limit=1G -f /var/www/nextcloud/cron.php
And get an email everyday:
PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 20814848728 bytes) in /var/www/nextcloud/lib/private/Files/Storage/Local.php on line 327
I'm running the current 28.0.1
Any suggestion howto futher debug what job exactly caused the issue?
Since it happens once every day at a specific timestamp it should be one of those:
Steps to reproduce
Create Crontab with the current release:
*/15 * * * * php8.1 -d memory_limit=1G -f /var/www/nextcloud/cron.php
Once per day, cron.php grows to 20GB Memory --> OOM
Expected behavior
Cronjob should not run OOM
Installation method
None
Nextcloud Server version
28
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.1
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 22 to 23)
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
***REMOVED SENSITIVE VALUE***
],
"default_phone_region": "DE",
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwrite.cli.url": "https:\/\/cloud.xxxx.com",
"dbtype": "mysql",
"version": "28.0.1.1",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"logfile": "\/var\/log\/nextcloud.log",
"loglevel": 0,
"log_rotate_size": "104857600",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"forcessl": true,
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_smtpmode": "sendmail",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"theme": "",
"maintenance": false,
"secret": "***REMOVED SENSITIVE VALUE***",
"updatechecker": false,
"trashbin_retention_obligation": "auto",
"appstoreurl": "https:\/\/apps.nextcloud.com\/api\/v1",
"appstoreenabled": true,
"mysql.utf8mb4": true,
"filelocking.enabled": "true",
"memcache.locking": "\\OC\\Memcache\\Redis",
"memcache.local": "\\OC\\Memcache\\Redis",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 6379,
"timeout": 0,
"password": "***REMOVED SENSITIVE VALUE***"
},
"updater.release.channel": "stable",
"app_install_overwrite": [
"bookmarks"
],
"memories.exiftool": "\/var\/www\/owncloud\/apps\/memories\/bin-ext\/exiftool-amd64-glibc",
"memories.vod.path": "\/var\/www\/owncloud\/apps\/memories\/bin-ext\/go-vod-amd64"
}
}List of activated Apps
Enabled:
- activity: 2.20.0
- audioplayer: 3.4.1
- bookmarks: 13.1.3
- bruteforcesettings: 2.8.0
- calendar: 4.6.3
- circles: 28.0.0-dev
- cloud_federation_api: 1.11.0
- comments: 1.18.0
- contacts: 5.5.1
- contactsinteraction: 1.9.0
- dashboard: 7.8.0
- dav: 1.29.1
- federatedfilesharing: 1.18.0
- federation: 1.18.0
- files: 2.0.0
- files_automatedtagging: 1.18.0
- files_external: 1.20.0
- files_pdfviewer: 2.9.0
- files_reminders: 1.1.0
- files_sharing: 1.20.0
- files_trashbin: 1.18.0
- files_versions: 1.21.0
- keeweb: 0.6.15
- logreader: 2.13.0
- lookup_server_connector: 1.16.0
- mail: 3.5.2
- nextcloud_announcements: 1.17.0
- notes: 4.9.2
- notifications: 2.16.0
- oauth2: 1.16.3
- password_policy: 1.18.0
- photos: 2.4.0
- privacy: 1.12.0
- provisioning_api: 1.18.0
- recommendations: 2.0.0
- related_resources: 1.3.0
- richdocuments: 8.3.0
- serverinfo: 1.18.0
- settings: 1.10.1
- sharebymail: 1.18.0
- support: 1.11.0
- survey_client: 1.16.0
- systemtags: 1.18.0
- tasks: 0.15.0
- text: 3.9.1
- theming: 2.3.0
- twofactor_backupcodes: 1.17.0
- updatenotification: 1.18.0
- viewer: 2.2.0
- weather_status: 1.8.0
- workflowengine: 2.10.0
Disabled:
- admin_audit: 1.18.0
- checksum: 1.2.2 (installed 1.2.2)
- duplicatefinder: 1.1.4 (installed 1.1.4)
- encryption: 2.16.0
- files_accesscontrol: 1.18.0 (installed 1.18.0)
- files_retention: 1.16.0 (installed 1.16.0)
- files_rightclick: 0.15.1 (installed 1.6.0)
- firstrunwizard: 2.17.0 (installed 2.17.0)
- groupfolders: 16.0.1 (installed 16.0.1)
- memories: 6.2.2 (installed 6.2.2)
- polls: 6.0.1 (installed 6.0.1)
- previewgenerator: 5.4.0 (installed 5.4.0)
- spreed: 18.0.1 (installed 18.0.1)
- suspicious_login: 6.0.0
- twofactor_totp: 10.0.0-beta.2
- user_ldap: 1.19.0
- user_status: 1.8.1 (installed 1.8.1)Nextcloud Signing status
No errors have been found.Nextcloud Logs
{"reqId":"EgNErFpZbZCmFkP57K24","level":0,"time":"2024-01-14T15:45:09+00:00","remoteAddr":"","user":"--","app":"cron","method":"","url":"--","message":"Run OC\\Core\\BackgroundJobs\\GenerateMetadataJob job with ID 119428","userAgent":"--","version":"28.0.1.1","data":{"app":"cron"}}
{"reqId":"EgNErFpZbZCmFkP57K24","level":3,"time":"2024-01-14T15:47:26+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"Allowed memory size of 1073741824 bytes exhausted (tried to allocate 20814848728 bytes) at /var/www/nextcloud/lib/private/Files/Storage/Local.php#327","userAgent":"--","version":"28.0.1.1","data":{"app":"PHP"}}Additional info
No response