TextSAM-US: Text Prompt Learning for SAM to Accurately Segment Breast Tumor in Ultrasound Images

Overview

Method

Segmentation Results

Installation

This codebase is tested on Ubuntu 20.04.2 LTS with python 3.10. Follow the below steps to create environment and install dependencies.

• Setup conda environment (recommended).

# Create a conda environment
conda create -n textsam_eus python=3.10 -y

# Activate the environment
conda activate textsam_eus

# Install torch (requires version >= 2.1.2) and torchvision
# Please refer to https://pytorch.org/ if you need a different cuda version
pip install torch==2.1.2 torchvision==0.16.2 --index-url https://download.pytorch.org/whl/cu118

• Clone TextSAM-EUS code repository and install requirements

# Clone MaPLe code base
git clone https://github.com/HealthX-Lab/TextSAM-EUS

cd TextSAM-EUS/
# Install requirements

pip install -e .

Data preparation

• Download the dataset here.

• Place dataset under data like the following:

data/
|–– EUS/
|   |–– train/
|   |   |–– images/
|   |   |–– masks/
|   |–– val/
|   |   |–– images/
|   |   |–– masks/
|   |–– test/
|   |   |–– images/
|   |   |–– masks/

Training and Evaluation

• Run the training and evaluation script

bash scripts/pipeline.sh EUS outputs

• You can change some design settings in the config.

Citation

If you use our work, please consider citing:

## Acknowledgements

Our code builds upon the [open_clip](https://github.com/mlfoundations/open_clip), [segment-anything](https://github.com/facebookresearch/segment-anything), and [MaPLe](https://github.com/muzairkhattak/multimodal-prompt-learning) repositories. We are grateful to the authors for making their code publicly available. If you use our model or code, we kindly request that you also consider citing these foundational works.