Trackastra - Tracking by Association with Transformers

Trackastra is a cell tracking approach that links already segmented cells in a microscopy timelapse by predicting associations with a transformer model. It comes with pretrained models that perform well out of the box for many types of live-cell imaging data.

Give it a try with our easy-to-use napari plugin, no parameters to tune!

Tracking results

Nuclei tracking	Bacteria tracking

Error visualisations made with Divisualisation

🏆 Trackastra (EPFL-CH) won the 7th Cell Tracking Challenge held at ISBI 2024, focusing on generalizable cell linking. For more results, please refer to our paper.

Installation

This repository contains the Python implementation of Trackastra. If you use Trackastra in your research, please cite our paper (BibTeX).

Please first set up a Python environment (with Python version 3.10 or higher), preferably via conda or mamba.

Simple installation

Trackastra can then be installed from PyPI using pip:

pip install trackastra

With ILP support

For tracking with an integer linear program (ILP, which is optional)

conda create --name trackastra python=3.10 --no-default-packages
conda activate trackastra
conda install -c conda-forge -c gurobi -c funkelab ilpy
pip install "trackastra[ilp]"

🆕😎 With pretrained features

For our new model variant that uses SAM2 features for improved tracking performance on certain data, for example for tracking bacteria:

pip install "trackastra[etultra]"

and select the general_2d_w_SAM2_features pre-trained model for predictions, preferably on a machine with a GPU (slow on CPU!).

Installation with training support

pip install "trackastra[train]"

📄

Development installation

conda create --name trackastra python=3.10 --no-default-packages
conda activate trackastra
conda install -c conda-forge -c gurobi -c funkelab ilpy
git clone https://github.com/weigertlab/trackastra.git
pip install -e "./trackastra[all]"

📄

Notes/Troubleshooting

• For the optional ILP linking, this will install motile and binaries for two discrete optimizers:

  1. The Gurobi Optimizer. This is a commercial solver, which requires a valid license. Academic licenses are provided for free, see here for how to obtain one.

  2. The SCIP Optimizer, a free and open source solver. If motile does not find a valid Gurobi license, it will fall back to using SCIP.

• On MacOS, installing packages into the conda environment before installing ilpy can cause problems.

• 2024-06-07: On Apple M3 chips, you might have to use the nightly build of torch and torchvision, or worst case build them yourself.

Usage: Tracking with a pretrained model

The input to Trackastra is a sequence of images and their corresponding cell (instance) segmentations. The available pretrained models are described in detail here.

Tracking with Trackastra can be done via:

Napari plugin

For a quick try of Trackastra on your data, please use our napari plugin, which already comes with pretrained models included.

Python API

All you need are the following two numpy arrays:

• imgs: a microscopy time lapse of shape time,(z),y,x.
• masks: corresponding instance segmentation of shape time,(z),y,x.

The predicted associations can then be used for linking with several modes:

• greedy_nodiv (greedy linking with no division) - fast, no additional dependencies
• greedy (greedy linking with division) - fast, no additional dependencies
• ilp (ILP based linking) - slower but more accurate, needs motile

Apart from that, no hyperparameters to choose :)

📄 Show python example

import torch
from trackastra.model import Trackastra
from trackastra.tracking import graph_to_ctc, graph_to_napari_tracks, write_to_geff
from trackastra.data import example_data_bacteria

device = "automatic" # explicit choices: [cuda, mps, cpu]

# load some test data images and masks
imgs, masks = example_data_bacteria()

# Load a pretrained model
model = Trackastra.from_pretrained("general_2d", device=device)

# or from a local folder
# model = Trackastra.from_folder('path/my_model_folder/', device=device)

# Track the cells
track_graph, masks_tracked = model.track(imgs, masks, mode="greedy")  # or mode="ilp", or "greedy_nodiv"

# Relabel the masks and write to cell tracking challenge format (CTC), 
ctc_tracks, ctc_masks = graph_to_ctc(
    track_graph,
    masks_tracked,
    outdir="tracked_ctc",
)

# Or write to the graph exchange file format (GEFF)
write_to_geff(
    track_graph,
    masks_tracked,
    outdir="tracked_geff.zarr",
)

You then can visualize the tracks with napari:

# Visualise in napari
napari_tracks, napari_tracks_graph, _ = graph_to_napari_tracks(track_graph)

import napari
v = napari.Viewer()
v.add_image(imgs)
v.add_labels(ctc_masks)
v.add_tracks(data=napari_tracks, graph=napari_tracks_graph)

Fiji (via TrackMate)

Trackastra is one of the available trackers in TrackMate. For installation and usage instructions take a look at this tutorial.

Docker images

Some of our models are available as docker images on Docker Hub. Currently, we only provide CPU-based docker images.

Track within a docker container with the following command, filling the <VARIABLES>:

docker run -it -v <LOCAL_DATA_DIR>:/data -v <LOCAL_RESULTS_DIR>:/results bentaculum/trackastra-track:<MODEL_TAG> --input_test /data/<DATASET_IN_CTC_FORMAT> --detection_folder <TRA/SEG/ETC>"

📄 Show example with Cell Tracking Challenge model:

wget http://data.celltrackingchallenge.net/training-datasets/Fluo-N2DH-GOWT1.zip 
chmod -R 775 Fluo-N2DH-GOWT1
docker pull bentaculum/trackastra-track:model.ctc-linking.ilp 
docker run -it -v ./:/data -v ./:/results bentaculum/trackastra-track:model.ctc-linking.ilp --input_test data/Fluo-N2DH-GOWT1/01 --detection_folder TRA

Command Line Interface

After installing Trackastra, simply run in your terminal

trackastra track --help

to build a command for tracking directly from images and corresponding instance segmentation masks saved on disk as two series of TIF files.

Usage: Training a model on your own data

To run an example:

• Clone this repository and go into the scripts directory with cd trackastra/scripts.
• Download the Fluo-N2DL-HeLa dataset from the Cell Tracking Challenge into data/ctc.

Then run:

python train.py --config example_config.yaml

Generally, training data needs to be provided in the Cell Tracking Challenge (CTC) format, i.e. annotations are located in a folder containing one or several subfolders named TRA, with masks and tracklet information.

Reference

Paper: Trackastra: Transformer-based cell tracking for live-cell microscopy

@inproceedings{gallusser2024trackastra,
  title={Trackastra: Transformer-based cell tracking for live-cell microscopy},
  author={Gallusser, Benjamin and Weigert, Martin},
  booktitle={European conference on computer vision},
  pages={467--484},
  year={2024},
  organization={Springer}
}