NoCodeSeg: Deep segmentation made easy!

⚠️Latest: Generic multiclass support has been added to the pipeline!

This is the official repository for the manuscript "Code-free development and deployment of deep segmentation models for digital pathology", published open access in Frontiers in Medicine.

The repository contains trained deep models for epithelium segmentation of HE and CD3 immunostained WSIs, as well as source code relevant for importing/exporting annotations/predictions in QuPath, both from DeepMIB and FastPathology.

All relevant scripts for working with our pipeline can be found in the source directory.

See here for how to access the trained models.

See here for how to download the 251 annotated WSIs.

Getting started

.

A video tutorial of the proposed pipeline was published on YouTube. It demonstrates the steps for:

• Downloading and installing the software
• QuPath
  • Create a project, then export annotations as patches with label files
  • Export patches from unannotated images for prediction in DeepMIB
  • (later) Import predictions for MIB and FastPathology as annotations
• MIB
  • Use the annotated patches/labels exported from QuPath
  • Configuring and training deep segmentation models (i.e. U-Net/SegNet)
  • Use the trained U-Net to predict unannotated patches exported from QuPath
  • Export trained models into the ONNX format for use in FastPathology
• FastPathology
  • Importing and creating a configuration file for the DeepMIB exported ONNX model
  • Create a project and load WSIs into a project
  • Use the U-Net ONNX model to render predictions on top of the WSI in real time
  • Export full sized WSI tiffs for import into QuPath

Note that the version of FastPathology used in the demonstration was v0.2.0 (this exact version can be downloaded from here). The software is continuously in development, and features presented in the video are therefore prone to changes in the near future. To get information regarding changes and new releases, please, visit the FastPathology repository.

Data

The 251 annotated WSIs are made openly available for anyone on DataverseNO. Alternatively, the data can be downloaded directly from Google Drive (click here to access the dataset). Information on how to cite the IBDColEpi dataset can be found on DataverseNO.

Reading annotations

The annotations are stored as tiled, pyramidal TIFFs, which makes it easy to generate patches from the data without the need for any preprocessing. Reading these files and working with them to generate training data, is already described in the tutorial video above.

TL;DR: Load TIFF as annotations in QuPath using provided groovy script and exporting these as labelled tiles.

Reading annotation in Python

If you wish to use Python, the annotations can be read exactly the same way as regular WSIs (for instance using pyFAST).

I have made a Jupyter Notebook demonstrating how to do this here.

Alternatively, click the CoLab button to access the notebook:

Models

Note that the trained models can only be used for academic purposes due to MIB's license. Trained model files (.mibDeep for MIB and .onnx for FastPathology) are made openly available on Google Drive. Simply download the file "trained-models.zip" and uncompress to get access the respective files.

Applications of pipeline

• Chiou et al., An immunohistochemical atlas of necroptotic pathway expression (2024), EMBO Molecular Medicine, https://doi.org/10.1038/s44321-024-00074-6
• Alharbi et al., A Deep Learning–Based Approach to Estimate Paneth Cell Granule Area in Celiac Disease (2023), Arch Pathol Lab Med, https://doi.org/10.5858/arpa.2023-0074-OA
• Mallén et al., Sex Differences in Glomerular Lesions, in Atherosclerosis Progression, and in the Response to Angiotensin-Converting Enzyme Inhibitors in the ApoE−/− Mice Model (2023), International Journal of Molecular Science, https://doi.org/10.3390/ijms241713442
• Røyset et al., Deep learning-based image analysis reveals significant differences in the number and distribution of mucosal CD3 and γδ T cells between Crohn's disease and ulcerative colitis, The Journal of Pathology, https://doi.org/10.1002/cjp2.301
• Pettersen et al., Code-free development and deployment of deep segmentation models for digital pathology (2022), Frontiers in Medicine, https://doi.org/10.3389/fmed.2021.816281

How to cite

Please, consider citing our paper, if you find the work useful:

@article{pettersen2022codefree,
    title={{Code-Free Development and Deployment of Deep Segmentation Models for Digital Pathology}},
    author={Pettersen, Henrik Sahlin and Belevich, Ilya and Røyset, Elin Synnøve and Smistad, Erik and Simpson, Melanie Rae and Jokitalo, Eija and Reinertsen, Ingerid and Bakke, Ingunn and Pedersen, André},         
    journal={Frontiers in Medicine},      
    volume={8},      
    year={2022},      
    url={https://www.frontiersin.org/article/10.3389/fmed.2021.816281},       
    doi={10.3389/fmed.2021.816281},      
    issn={2296-858X}
}

In addition, if you used the data set in your work, please, cite the following:

@data{pettersen2021ibdcolepi,
    title = {{140 HE and 111 CD3-stained colon biopsies of active and inactivate inflammatory bowel disease with epithelium annotated: the IBDColEpi dataset}},
    author = {Pettersen, Henrik Sahlin and Belevich, Ilya and Røyset, Elin Synnøve and Smistad, Erik and Jokitalo, Eija and Reinertsen, Ingerid and Bakke, Ingunn and Pedersen, André},
    publisher = {DataverseNO},
    year = {2021},
    version = {V2},
    doi = {10.18710/TLA01U},
    url = {https://doi.org/10.18710/TLA01U}
}

Acknowledgements

We wish to give our praise to Peter Bankhead and the QuPath team for their continuous support and assistance with QuPath and for assisting us in developing the scripts related to this study.