Nextpie is a reporting tool for Nextflow workflows. It processes run metadata (trace files) generated by Nextflow, stores them in a database, and enables users to perform aggregate analyses. The tool is built using Python Flask and uses a UI design provided by AppSeed.
Nextpie comes preloaded with a sample database, allowing users to run it with Docker without needing any initial configuration. This makes it easy to evaluate the tool without a complex setup.
If you plan to deploy Nextpie in a production environment, please make sure to clear the database and configure SMTP settings via the web interface: Settings → SMTP Settings.
NOTE: Starting from version
>=0.0.2, Nextflow pipelines require the nf-nextpie plugin (a client) to upload resource usage data to Nextpie. Users no longer need to modify their pipeline code to integrate with Nextpie. Data is automatically uploaded to Nextpie when the pipeline completes. The example pipeline offers more details on using the plugin.
Nextpie has been tested with Python v3.9 on Red Hat Enterprise Linux 9 and Ubuntu 24.04 LTS. All Python packages are version-pinned in the requirements directory to ensure reproducibility. As long as you can set up a Python virtual environment with the specified versions, it should work on most Linux distributions.
- Linux operating system
- Python == 3.9 (with virtual environment)
- Nextflow >= 24.10.4
- Conda
- Guix
- Docker
- Gunicorn
Nextflow's -resume command-line option allows rerunning a previously failed pipeline by reusing cached results from earlier runs.
Nextpie uses an SQLite database to store each process listed in the trace file. It uniquely identifies each process using a combination of the process hash (generated by Nextflow), the research group name, and the research project name. When a process fails, Nextflow assigns a new hash to it. For processes that complete successfully, however, Nextflow retains the same hash. As a result, Nextpie avoids inserting duplicate entries into the database by recognizing that a completed process has already been recorded during a previous pipeline execution—regardless of whether the -resume option is used. This deduplication mechanism works only when both the research group and project names are explicitly provided.
Although Nextpie is designed to expect a research group and project name, these can be omitted when running a pipeline (versions > 0.0.1). This added flexibility is intentional. In the absence of these identifiers, Nextpie automatically generates random names for both the research group and the research project to fill in the missing information. However, because these randomly generated names are not consistent across runs, Nextpie cannot determine whether a process belongs to the same project as a previous one. As a result, every pipeline run—regardless of the use of the -resume option—is treated as a completely new execution. Consequently, all process entries from the trace file are inserted into the database as new records.
NOTE: The test data contains
CACHEDprocesses because the trace files for the test data was generated using relatively older version of Nextflow. Old Nextflow versions gave different hashes to completed and cached processes.
- Python virtual environment (Development) [Instructional video]
- Conda environment [Instructional video]
- Guix environment
- Docker [Instructional video]
- Gunicorn
- Waitress (Windows)
- Run an Example Nextflow Workflow [Demo video]
- Run Nextpie on a Non-Default Port
- Nextpie API Documentation
- Admin Interface