Skip to content

Docker: deprecate docker.userEmulation and use docker.runOptions; mitigates Trim Galore pipe freeze #435

@yxkwgp

Description

@yxkwgp

Description of the bug

Summary

When running the pipeline with -profile docker, two related issues occur:

  1. Deprecated config: Nextflow logs a warning that docker.userEmulation is no longer supported and should be removed from the config.
  2. Trim Galore can freeze: The Trim Galore step sometimes hangs in the cutadapt stage when run inside Docker. This is a known upstream issue (Trim Galore #207) caused by small pipe buffers in container environments.

Root cause (Trim Galore freeze)

  • By default, Docker containers run as root (UID 0). The Linux kernel accounts pipe buffer usage per UID.
  • When the total pipe usage for root exceeds pipe-user-pages-soft, new pipes are limited to 2 pages (8192 bytes) instead of the default 16 pages (~64 KB).
  • With many Nextflow tasks (each creating pipes, e.g. cutadapt ↔ trim_galore), root’s quota is easily exceeded, so new pipes get the small size and Trim Galore can block waiting for the last few lines from cutadapt (see Trim Galore #207).
  • Running containers as the host user (non-root) spreads pipe usage by UID and often avoids hitting that limit, which can mitigate or avoid the freeze.

Current config (problematic)

In nextflow.config, the docker profile uses the deprecated option:

docker {
docker.enabled = true
docker.userEmulation = true // deprecated, no longer supported by Nextflow
...
}

Suggested change

Remove docker.userEmulation = true from the docker profile (to clear the warning and stop relying on removed behaviour).
Add docker.runOptions = '-u $(id -u):$(id -g)' to the docker profile so containers run as the current user.

Command used and terminal output

nextflow -bg -log test.log -q run ~/pipeline/nf-core/atacseq-2.1.2 -profile docker -params-file params.json -resume

Relevant files

No response

System information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions