How to generate the patch_labels for CAMELYON16?

[Rainydu184]

CLAM processes WSIs by generating a coords file and then extracts features into a features file based on the H5 file. How should I generate the corresponding patch_labels .npy file? Alternatively, could you upload a sample .npy file as a reference for the expected format?

[Franblueee]

Hi!

I have just uploaded the script we used to generate the patch labels:
https://github.com/Franblueee/SmMIL/blob/main/code/experiments/extract_patch_labels.py

Let me know if you need anything else.

Fran

[Rainydu184]

Thanks for your reply!

[Rainydu184]

Hello！

I encountered some issues while conducting experiments with COMELYON16.
When using the code you provided, it by default also preprocesses the COMELYON16 data. However, you mentioned that this dataset does not require additional processing, so I skipped the preprocessing step by setting the processed parameter for COMELYON16 to True in ProcessedMILDataset.

Yet, I am still facing some issues. Interestingly, it loads the features extracted by CLAM as .npy files, whereas they should be .pt files. Did you use a different data loading code for the COMELYON16 dataset? Below is the error message I received:

[Franblueee]

Hi,

We used .npy files since loading data was faster that way. You should convert the .pt files generated by CLAM to .npy files (you can create a simple script in python that loads the .pt file, extracts the array in it, and then saves it as .npy). Alternatively, you can modify the CLAM script so it saves the data directly in .npy format.

Fran

[Rainydu184]

Hi,

I have successfully run the code on the C16 dataset, but I'm not entirely sure if my parameter settings are correct. I obtained the following results: I used the repo on ABMIL and achieved a test set AUC of 0.930, but on sm_abmil the results seem to be worse. Could I have made a mistake somewhere? Do you have any suggestions?

This is shell code I have used:

Here is result:

rainydu

[Franblueee]

Hi,

Sorry for the late reply. I have been running the code, here are the results I get with your same configuration:

Also a snapshot of the run with seed=1:

I think you are not using the right features. Resnet50 trained with Barlow Twins (resnet50_bt) should have 2048 features per instance. Note that in my snapshot you can read Input dim: 2048, while in yours Input dim: 1024.

We downloaded the weights from here: https://github.com/lunit-io/benchmark-ssl-pathology/tree/main.

Fran

[Rainydu184]

Hello,

Thank you very much for your kind and helpful response. I’ve conducted several more experiments and finally achieved some promising results.

Since I didn’t use the CLAM method for feature extraction, the instance features did not align with the coordinates in the h5 files. Additionally, there was an issue with incorrect feature dimensions. After correcting these problems, I was able to obtain the following results.

I now have some new ideas I’d like to explore, and I also hope to use your method as a baseline for comparison. However, my current results still haven’t reached the level reported in your paper, so I’m continuing to test other models provided in your code. I suspect that the way the slides are processed—or possibly the instance-level labels—might play a significant role.

To help ensure a more accurate reproduction of your results, would it be possible for you to share the h5 files and the pseudo labels you used?

Once again, I sincerely appreciate all the support and resources you’ve already provided!

rainydu

[Franblueee]

Hi!

We have just open sourced torchmil, a pytorch-based library for deep multiple instance learning. You can check it out here: https://franblueee.github.io/torchmil/ maybe it is useful for you. Any feedback/contribution is appreciated.

In addition to several models and utilities, we have also released the datasets we used in this paper. Specifically, the CAMELYON16 dataset can be found here: https://huggingface.co/datasets/Franblueee/Camelyon16_MIL. You can use this dataset within torchmil: https://franblueee.github.io/torchmil/api/datasets/camelyon16mil_dataset/. These are the files I used to obtain the results in my previous response.

Hope this helps!

Fran