Nuclear size modification in script

[megleila]

I am trying out using Mesmer for segmentation through the Bodenmiller steinbock/IMCRtools pipeline and applying it on human kidney tissue samples. I am finding that using the default training data for Mesmer is resulting in very small (1-2 pixel in size) debris being identified as nuclei. This happens both for cell segmentation and for nuclear segmentation. Looking at the source code available online (https://deepcell.readthedocs.io/en/latest/_modules/deepcell/applications/mesmer.html), I wondered whether you would recommend adjusting any of these parameters (ie nuclear radius) for this issue, or if you had another suggestion I could try? Is this a common issue?

Wondered if you had a suggested alteration to source code or training data to use to improve inappropriate labeling of debris as nuclei. We previously segmented using Ilastik/CellProfiler, but are looking for a more streamlined approach -- and additionally the segmentation of true nuclei does appear more accurate in DeepCell than it does in our former approach.

Deepcell segmentation including many very small debris identified as cells/nuclei

Nuclear marker only

[ngreenwald]

Hey @megleila, I would first try submitting the same image to deepcell.org/predict. The issue you're raising should be addressed by the default parameters, so I want to check and see if something specific about how the model is being run within steinbock

[megleila]

When I submit my image for Mesmer on deepcell.org/predict setting the resolution appropriately to be 10x 1um/pixel, it seems to be the same result or at least definitely the same issue — although I’ve tried everything and I can only see segmentation of the very top ribbon of the image using this site. The size of the image is 1500x700, so not bigger than allowed by the site.

I tried running the program at 20x (0.5um/pixel) which is not the accurate resolution, and the smaller nuclei are fewer in number but some of the other segmentation looks a little off. Nuclear outlines still cut off or incompletely done for whatever reason as you can see.

Raw image I imported to deepcell.org/predict:

[ngreenwald]

Got it, thanks for the detailed response.

I'm not sure why the display is only loading the first set of segmentations, perhaps someone else (@rossbar?) could chime in. I remember this issue popped up before when there was a limit set on the number of segmentations that could be displayed at once, since they're numbered sequentially from the top.

I think I figured out the issue with small cells. The post-processing function we use for mesmer is here. There's an argument, small_objects_threshold, which controls the minimum size of a cell in order to be included. This is set in the kwargs dict here. However, I think these settings apply to the resized image, not the input image. So the default of 15 pixels becomes 4 pixels when resized from 20X (what the model is run at) to 10X (what is being returned). Hence you end up getting cells which are much smaller than desired. You should be able to change this default setting if you're calling the function directly.

[megleila]

I really appreciate this detailed response. Since your response a week ago we have been looking into this alongside our lab’s main bioinformatics research scientist, and are still figuring this out for our data. I wanted to respond to acknowledge that I (immediately) read your response, and our appreciation of your quick response prior to getting back to you with whether these suggestions worked! On Jul 10, 2023, at 6:46 PM, Noah F. Greenwald ***@***.***> wrote: Got it, thanks for the detailed response. I'm not sure why the display is only loading the first set of segmentations, perhaps someone else ***@***.***<https://github.com/rossbar>?) could chime in. I remember this issue popped up before when there was a limit set on the number of segmentations that could be displayed at once, since they're numbered sequentially from the top. I think I figured out the issue with small cells. The post-processing function we use for mesmer is here<https://github.com/vanvalenlab/deepcell-toolbox/blob/master/deepcell_toolbox/deep_watershed.py#L45>. There's an argument, small_objects_threshold, which controls the minimum size of a cell in order to be included. This is set in the kwargs dict here<https://github.com/vanvalenlab/deepcell-tf/blob/master/deepcell/applications/mesmer.py#L269>. However, I think these settings apply to the resized image, not the input image. So the default of 15 pixels becomes 4 pixels when resized from 20X (what the model is run at) to 10X (what is being returned). Hence you end up getting cells which are much smaller than desired. You should be able to change this default setting if you're calling the function directly. — Reply to this email directly, view it on GitHub<#672 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BAH2R6MB6SA6O7BQOVGDMQLXPSA5ZANCNFSM6AAAAAA2A2HYU4>. You are receiving this because you were mentioned.Message ID: ***@***.***>