subflow

1. Workflow Overview
2. Installation
3. Usage
4. Troubleshooting
5. License

Workflow Overview

↓ on-the-fly

• Monitors incoming files for on-the-fly processing of movies, micrographs, and particles.

• Aligns movie frames and estimates CTF values using Relion schemes (Burt et al., 2024).

• Monitors preprocessed micrographs and picks filaments using crYOLO (Wagner et al., 2020).

• Subtracts those filaments from micrographs using the lattice-subtraction scripts (Chai et al., 2022).

• Filaments are split before subtraction to reduce artefacts induced by bending.

• Allows for two rounds of filament picking and subtraction if requested.

• Picks particles from subtracted micrographs using crYOLO (Wagner et al., 2019).

• Allows viewing of micrographs and picked coordinates (if available) to monitor progress.

↑ on-the-fly

• Prepares Relion .star files with the modified paths to the subtracted micrographs.

• Extracts particles, imports them to cryoSPARC (Punjani et al., 2017).

• Performs heterogeneous refinement in cryoSPARC with user-defined volumes to extract a good class.

• Converts the particles back to Relion .star format using pyem (Asarnow et al., 2019).

Installation

• Set up a fresh conda environment with Python >= 3.9: conda create -n subflow python=3.9

• Activate the environment: conda activate subflow

• Install subflow: pip install subflow

• Follow the instructions to install Relion and cryoSPARC

• Install crYOLO 1.9.1 and pyem into their own anaconda environments

• Download the multi-curve-fitting and filament subtraction scripts.

• Edit the config.json file in the subflow installation to point to the correct crYOLO and pyem files, as well as the multi-curve-fitting and subtraction scripts

Usage

Start the GUI by running the following with the conda environment active:

subflow &

• After saving parameters to a file (e.g. subflow.txt), re-opening the GUI at a later date opens the previous parameters. Saving is not automatic.

• Tabs with hollow circles can be run on-the-fly (i.e. monitor files from the previous step). They turn into a black dot once they are running.

• Clicking twice on a tab toggles the following * display tab to view results for that step.

Tab Order

1. SUBFLOW — Global settings: config path, EER toggle, Subtract twice toggle, load/save parameters, start/stop all.
2. Link mov — Link incoming movies into the working directory.
3. eer to tif — Convert EER stacks to TIFF (run if using EER).
4. Preproc — Relion import, motion correction, and CTF estimation.
5. Link mic — Link corrected micrographs into Micrographs.
6. * (Display micrographs) — QC linked micrographs.
7. Pick fil — crYOLO filament picking on micrographs.
8. * (Display picks) — QC filament picks.
9. Fit curves — Multi-curve fitting on filament picks.
10. * (Display fit) — QC fitted curves.
11. Split fil — Split fitted filaments into shorter segments.
12. * (Display splits) — QC split results.
13. Subtract — Lattice subtraction using split filaments.
14. * (Display subtraction) — QC subtracted micrographs (optionally overlay split coords).
15. Pick comp — crYOLO picking of complexes on subtracted micrographs.
16. * (Display comp picks) — QC complex picks.

The next block appears only when Subtract twice is enabled in SUBFLOW:

1. Pick fil (round 2) — Filament picking on the first subtraction.

2. * (Display picks) — QC filament picks (round 2).

3. Fit curves (round 2) — Curve fitting (round 2).

4. * (Display fit) — QC fit results (round 2).

5. Split fil (round 2) — Split filaments (round 2).

6. * (Display splits) — QC split results (round 2).

7. Subtract (round 2) — Second subtraction pass.

8. * (Display subtraction) — QC subtracted micrographs (round 2).

9. Pick comp (round 2) — Complex picking on round-2 subtractions.

10. * (Display comp picks) — QC complex picks (round 2).

11. Merge — Merge sequential subtractions and/or complex picks (used with double subtraction).

12. Prep star — Fix micrograph .star paths to subtracted micrographs and link picks into that directory.

13. Extract — Relion import + extract particles from the subtracted micrographs.

14. Hetero — cryoSPARC workspace, import particles/volumes, heterogeneous refinement.

Troubleshooting

• If Subflow crashes while Relion jobs are running in the Preproc tab, they will continue to run. You can check this by checking the Relion GUI. You may need to kill those jobs before being able to run Preproc again, or just continue in the Relion GUI.

• Since cryosparc-tools is set to version 4.6.0 in the installation, your cryoSPARC installation must be 4.6.0. Otherwise cryosparc-tools needs to be downgraded/upgraded.

License

This project is licensed under the MIT License - see the LICENSE.txt file for details.