Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable cleanup on remote work dir #5742

Merged
merged 3 commits into from
Feb 8, 2025
Merged

Disable cleanup on remote work dir #5742

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eaf0ebd
Disable cleanup on remote work dir
pditommaso Feb 4, 2025
b8b0404
Apply suggestions from code review
bentsherman Feb 5, 2025
1156c1f
Update docs/reference/config.md [ci skip]
pditommaso Feb 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/reference/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ This page lists all of the available settings in the {ref}`Nextflow configuratio
: If `true`, on a successful completion of a run all files in *work* directory are automatically deleted.

:::{warning}
The use of the `cleanup` option will prevent the use of the *resume* feature on subsequent executions of that pipeline run. Also, be aware that deleting all scratch files can take a lot of time, especially when using a shared file system or remote cloud storage.
The use of the `cleanup` option will prevent the use of the *resume* feature on subsequent executions of that pipeline run.
:::
:::{warning}
The `cleanup` option is not supported for remote work directories, such as Amazon S3, Google Cloud Storage, and Azure Blob Storage.
:::

`dumpHashes`
Expand Down
5 changes: 5 additions & 0 deletions modules/nextflow/src/main/groovy/nextflow/Session.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1209,6 +1209,11 @@ class Session implements ISession {
if( aborted || cancelled || error )
return

if( workDir.scheme != 'file' ) {
log.warn "The `cleanup` option is not supported for remote work directory: ${workDir.toUriString()}"
return
}

log.trace "Cleaning-up workdir"
try (CacheDB db = CacheFactory.create(uniqueId, runName).openForRead()) {
db.eachRecord { HashCode hash, TraceRecord record ->
Expand Down