monailabel/monailabel_HelloWorld_radiology_3dslicer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (c) MONAI Consortium  \n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
    "you may not use this file except in compliance with the License.  \n",
    "You may obtain a copy of the License at  \n",
    "&nbsp;&nbsp;&nbsp;&nbsp;http://www.apache.org/licenses/LICENSE-2.0  \n",
    "Unless required by applicable law or agreed to in writing, software  \n",
    "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
    "See the License for the specific language governing permissions and  \n",
    "limitations under the License. \n",
    " \n",
    "# MONAI Label Hello, World! - Radiology App and Slicer Example\n",
    "\n",
    "***The Active Learning Process with MONAI Label***\n",
    "\n",
    "In this notebook, we provide a hello world example of MONAI Label use case. Using Spleen segmentation in Radiology app as the demonstration, 3D Slicer as the client viewer, we show how MONAI Label workflow serves as interacitve AI-Assisted tool for labeling CT scans. \n",
    "\n",
    "![workflow](./figures/monailabel_radiology_3dslicer/teaser_img.png)\n",
    "\n",
    "## 1. Prepare MONAI Label\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup environment\n",
    "\n",
    "### Prerequisites\n",
    "- **Install MONAI Label** weekly preview release: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install monailabel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Check Install and All Sample Apps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!monailabel apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Download radiology app\n",
    "Choose \"radiology\" as the app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!monailabel apps --download --name radiology --output apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 Download sample data\n",
    "Download MSD Spleen dataset as the sample dataset using monailabel API. The task is the volumetric (3D) segmentation of the spleen from CT image. The segmentation of spleen is formulated as the voxel-wise 2-class classification. Each voxel is predicted as either foreground (spleen) or background. The dataset is from the 2018 MICCAI challenge.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!monailabel datasets --download --name Task09_Spleen --output datasets"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 Starting MONAI Label Server\n",
    "\n",
    "\"The `segmentation_spleen` model will be automatically downloaded upon starting the MONAI Label server. The network and checkpoint will be processed during the initialization steps of the `monailabel` command.\n",
    "\n",
    "**Note:** If you are running a MONAI Label server in this notebook and plan to close the notebook, ensure that you terminate the kernel first. Failing to do so might result in an issue where the port remains occupied from the previous session when you try to restart the server later. To prevent this, always terminate the kernel before closing the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!monailabel start_server --app apps/radiology --studies datasets/Task09_Spleen/imagesTr --conf models segmentation_spleen"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Auto Segmentation with 3D Slicer\n",
    "\n",
    "Below is a step-by-step tutorial on using 3D Slicer for auto spleen segmentation. Pre-trained model is provided and loaded by MONAI Label server.\n",
    "\n",
    "### Installation of 3D Slicer and MONAI Label Plugin\n",
    "\n",
    "You can download 3D Slicer on the [3D Slicer download page](https://download.slicer.org/).\n",
    "MONAI Label supports stable and preview versions of 3D Slicer: [Stable and Preview Version >= 5.0](https://download.slicer.org/).\n",
    "\n",
    "**For Ubuntu**\n",
    "\n",
    "**Prerequisites for 3D Slicer**: Slicer is expected to work on the vast majority of desktop and server Linux distributions. The system is required to provide at least GLIBC 2.17 and GLIBCCC 3.4.19. For more details, please refer to [prerequisites](https://slicer.readthedocs.io/en/latest/user_guide/getting_started.html#linux). \n",
    "\n",
    "As an example, Ubuntu users can click the downloaded 3D Slicer file and extract to the working folder, then click \"Slicer\" to open the 3D Slicer viewer. Specifically, Open the tar.gz archive and copy directory to the location of your choice. Installation of additional packages may be necessary depending on the Linux distribution and version. Run the Slicer executable to start the 3D Slicer.\n",
    "\n",
    "**For Windows**\n",
    "\n",
    "Download and run the 3D Slicer installer. Once installed, you can run `Slicer` from the Windows Start Menu.\n",
    "\n",
    "#### Install MONAI Label plugin from Slicer Extension Manager\n",
    "\n",
    "**Step 1:** Start 3D Slicer and navigate to **View** -> **Extension Manager** -> **Active Learning** -> **MONAI Label**\n",
    "\n",
    "<img src=\"./figures/monailabel_radiology_3dslicer/3dslicer-extensions-manager.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n",
    "**Step 2:** Install the MONAI Label plugin (click \"install\") and restart 3D Slicer. The \"Restart\" button is in the lower right-hand corner of the Extension Manager dialog box.\n",
    "\n",
    "**Step 3:** Add shortcut: Navigate to **Edit** -> **Application Settings**, under the **Modules** panel drag **MONAI Label** into the **Favorite Modules** panel. Click **OK**, restart 3D Slicer, and the MONAI Label icon <img src= \"./figures/monailabel_radiology_3dslicer/MONAILabel.png\" alt=\" drawing\" width=\" 20\"/> will be visible in the toolbar.\n",
    "\n",
    "<img src=\"./figures/monailabel_radiology_3dslicer/monai-label-plugin-favorite-modules-1.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n",
    "\n",
    "### 2.1 Start 3D Slicer and MONAI Label Plugin\n",
    "\n",
    "- On the menu bar navigate click MONAI Label, connect and Check MONAI Label Server Address/Name, radiology app, and confirm 3D Slicer is connected to MONAI Label server correctly, if no, fill in the correct server address and port.\n",
    "\n",
    "![slicer1](./figures/monailabel_radiology_3dslicer/slicer1.png)\n",
    "\n",
    "\n",
    "### 2.2 Select Model and Fetch Data\n",
    "\n",
    "- Select the model users would like to use, such as \"**segmentation_spleen**\" in this tutorial, click **next sample** to load image data from study datastore.\n",
    "\n",
    "![slicer1](./figures/monailabel_radiology_3dslicer/slicer2.png)\n",
    "\n",
    "\n",
    "### 2.3 Run Auto Segmentation\n",
    "\n",
    "- MONAI Label has provided pre-trained model weights for the spleen segmentation model, click **run** to do the inference on the currently loaded data. \n",
    "\n",
    "- Note: the pre-trained models will be downloaded when starting MONAI Label server in the \"**model**\" folder. For instance, in this use case, the downloaded pre-trained model is saved at \"**apps/radiology/model/pretrained_segmentation_spleen.pt**\".\n",
    "\n",
    "- Users may move the slices and location to view the automatic inferences segmentation.\n",
    "\n",
    "![slicer1](./figures/monailabel_radiology_3dslicer/slicer3.png)\n",
    "\n",
    "\n",
    "### 2.4 Edit Annotations and Submit Label\n",
    "\n",
    "- Users can edit the inference labels manually using Segment Editor, the tool for adjusting annotation. Once done annotation, click **Submit Label** to save the ground truth label to the file system.\n",
    "\n",
    "- Note: the final annotations will be saved to \"**labels/final**\" folder in the study dataset, for example, in this use case, the ground truth label will be saved to \"**datasets/Task09_Spleen/imagesTr/labels/final**\"\n",
    "\n",
    "#### AI assisted corrections:\n",
    "\n",
    "![scribbles](./figures/monailabel_radiology_3dslicer/scribbles.png)\n",
    "\n",
    "#### Corrections with Segment Editor:\n",
    "\n",
    "![slicer1](./figures/monailabel_radiology_3dslicer/slicer4.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Active Learning with 3D Slicer\n",
    "\n",
    "Active learning and interactive fine-tuning models have highlighted features in MONAI Label, users can train their models anytime when new annotated labels are saved. Click **Train** button, MONAI Label server will fetch saved final ground truth label and fine-tune the prior model. \n",
    "\n",
    "![slicer1](./figures/monailabel_radiology_3dslicer/slicer5.png)\n",
    "\n",
    "Users can do auto segmentation using the latest fine-tuned model. \n",
    "Active learning process will select unlabeld batch data.\n",
    "\n",
    "Next image selected, the already trained images will be marked as labeled data, then won't be selected for next labeling batch. Active learning strategies such as \"first/random\" will be used for selecting which unlabeled data to fetch. \n",
    "\n",
    "For general active learning workflow design, see the slide: ![activeL](./figures/monailabel_radiology_3dslicer/activeLearning.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Repeat the Interactive Labeling Process \n",
    "\n",
    "Repeated fetch data and active learning process to section 2.2, until all unlabeled data are annotated and trained. \n",
    "\n",
    "For each training loop, the new best metric model will be saved in \"**model/<model_name>.pt**\", in this use case, \"\"**apps/radiology/model/segmentation_spleen.pt**\" is saved. For detailed training stats and step models, refer to \"**apps/radiology/model/segmentation_spleen/train_01**\", "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Conclusion\n",
    "\n",
    "This tutorial demonstrates basic usage of MONAI Label with radiology app, a spleen segmentation task is introduced. Active learning process, auto segmentation, segmentation editor, submit labels, and save models are included. \n",
    "We highlight the interactive learning process when labeling a dataset, our platform can achieve incremental improvements and understand medical data better for researchers and clinicians. \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}