After deleting image, the preview files remain forever

[ho1ger]

This issue is obviously related to #5558, which was closed for Nextcloud 12. I run Nextcloud 13.0.6 and the issue exists for me. Can anyone confirm?

Steps to reproduce

1. Upload images (in my case I had about 5gb of JPEGs)
2. Let previews be generated (e.g. using the preview generator app and a cron job)
3. Delete those images
4. Purge trashbin
   (5. Wait for trashbin_retention_obligation is exceeded)

Expected behaviour

The cached previews should be deleted at some point in time...

Actual behaviour

The cached previews are NOT deleted, in my case I have 13GB of preview images of different sizes

Server configuration

Operating system:
Debian

Web server:
Nginx

Database:
Maria

PHP version:
7.2
Nextcloud version: (see Nextcloud admin page)
13.0.6

Updated from an older Nextcloud/ownCloud or fresh install:
Updated; but previews were generated on NC 13

List of activated apps:

Details

``` - activity: 2.6.1 - admin_audit: 1.3.0 - admin_notifications: 1.0.1 - apporder: 0.5.0 - bookmarks: 0.12.2 - bookmarks_fulltextsearch: 1.0.0 - bruteforcesettings: 1.1.0 - calendar: 1.6.1 - caniupdate: 0.1.3 - contacts: 2.1.5 - dav: 1.4.7 - federatedfilesharing: 1.3.1 - files: 1.8.0 - files_fulltextsearch: 1.0.0 - files_fulltextsearch_tesseract: 1.0.0 - files_pdfviewer: 1.2.1 - files_rightclick: 0.8.4 - files_sharing: 1.5.0 - files_texteditor: 2.5.1 - files_trashbin: 1.3.0 - files_versions: 1.6.0 - files_videoplayer: 1.2.0 - fulltextsearch: 1.0.0 - fulltextsearch_elasticsearch: 1.0.0 - gallery: 18.0.0 - impersonate: 1.0.4 - logreader: 2.0.0 - lookup_server_connector: 1.1.0 - metadata: 0.7.0 - nextcloud_announcements: 1.2.0 - notifications: 2.1.2 - oauth2: 1.1.1 - password_policy: 1.3.0 - previewgenerator: 1.1.0 - provisioning_api: 1.3.0 - serverinfo: 1.3.0 - sharerenamer: 1.5.0 - twofactor_backupcodes: 1.2.3 - twofactor_totp: 1.4.1 - updatenotification: 1.3.0 - workflowengine: 1.3.0 ```

Nextcloud configuration:

Details

``` { "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "xxxxxx", "xxxxxx" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "xxxxxxxxx", "dbtype": "mysql", "version": "13.0.6.1", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "memcache.local": "\\OC\\Memcache\\APCu", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_domain": "***REMOVED SENSITIVE VALUE***", "maintenance": false, "logtimezone": "Europe\/Berlin", "loglevel": 0, "theme": "", "trashbin_retention_obligation": "auto, 7", "updater.release.channel": "production", "mail_smtpauth": 1, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpsecure": "tls", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***" } ```

[nextcloud-bot]

GitMate.io thinks possibly related issues are #5558 (After deleting an (image) file, the preview files remain forever), #9523 (A file completely disappears after deleting from a group folder), #8241 (image), #2413 (Previews for empty files broken), and #3041 (Deleting a preview folder stops writing previews to disk).

[rullzer]

Can you check if you have an entry in your background jobs: OC\Preview\BackgroundCleanupJob?

Also note that if you use ajax cron it will only delete 10 at a time (to not run into timeouts).

[ho1ger]

I am using a Linux cron job, not Ajax cron.

Sorry; how do I access OC\Preview\BackgroundCleanupJob?

[ho1ger]

In the logs I found the following which is related to background Cleanup Jobs:

{"reqId":"0br9L4RsCRwi9HJhnVr9","level":1,"time":"2018-06-08T10:32:51+02:00","remoteAddr":"xxx","user":"--","app":"updater","method":"GET","url":"\/core\/ajax\/update.php?requesttoken=SvT6GXpQphnGbl9Mg9WdNOUBmJ35bJJ5GLpgNhsC%2FQk%3D%3AJ6yiexIZkHu8I24G6L6sYLBY0tyQAuU6XdE5cllluD0%3D","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.1 Safari\/605.1.15","version":"13.0.2.1"}
{"reqId":"JiMmzSdZG0ya4yWgnY5p","level":1,"time":"2018-06-12T07:45:32+02:00","remoteAddr":"","user":"--","app":"updater","method":"--","url":"--","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"--","version":"13.0.3.2"}
{"reqId":"Vuip1twG4c8HRKSb8r1M","level":1,"time":"2018-08-30T18:40:17+02:00","remoteAddr":"","user":"--","app":"updater","method":"--","url":"--","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"--","version":"13.0.4.0"}
{"reqId":"3N5R2TG63uE5RtzH9Hh1","level":1,"time":"2018-08-30T18:40:48+02:00","remoteAddr":"xxxxxx","user":"--","app":"updater","method":"GET","url":"\/core\/ajax\/update.php?requesttoken=Y9%2FT7HxGSiDjwrAVSLSFWrJQPYx4Lr9Dd44Qy4F6hmc%3D%3AULXl3R8pPGKRuoRsC4G1PcZ7CcVNVsUkJtdyr9VL1gU%3D","message":"\\OC\\Repair::step: Repair step: Add preview cleanup background jobs","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.2 Safari\/605.1.15","version":"13.0.4.0"}

[ho1ger]

Q: What would actually happen if I just delete the entire preview folder? Would the preview generator understand that some needed previews are missing and recreate them?

[rullzer]

AH sorry. Check the oc_jobs table.

No it won'tyou need to rescan your appdata folder with the occ command

[ho1ger]

Hope I got that right: select * from oc_jobs where class="OC\Preview\BackgroundCleanupJob"; does not return anything.

[nachoparker]

I can confirm this is still an issue in NC15.0.8

[Iolaum]

I have experienced the same issue in my installation of NextCloud on a nextcloudbox running nextcloudpi. I believe the problem lied with the preview generator app. When I realized the previews were eating up a huge amound of space I ended up deleting the app and manually removed the offending data.