Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: Enduro not cleaning up temporary files causing disk space to fill up #627

Open
joel-simpson opened this issue Aug 5, 2024 · 0 comments

Comments

@joel-simpson
Copy link

Enduro (0.41.7) seems to be filling up the processing directory with blobs and extract-folder.

Client is configuring [[pipeline]] → processingDir = /var/enduro/processing
The documentation does not state a default path for some reason.

On one of their submissions we preserved around 500GB worth of AIPs during 10 hours, starting with an empty processing directory. Enduro then crashed with a database error because it had filled the /var partition (934GB in /var/enduro/processing).

Temporary workaround put in place to run an hourly cron job, deleting everything older than 6 hours from /var/enduro/processing.

But wondering if enduro could clean up temporary files as part of the workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant