GTV segmentation in MRI guided radiotherapy with promptable foundation models.
Authors
Affiliations (11)
Affiliations (11)
- Radiation Oncology, University Hospital of Munich, Marchioninistr. 15, Munich, 81377, GERMANY.
- Department of Radiation Oncology, Amsterdam UMC, de Boelelaan 1117, Amsterdam, 1081 HV, NETHERLANDS.
- Department of Radiation Oncology, University Hospital Heidelberg, Im Neuenheimer Feld 400, 69120 Heidelberg, Heidelberg, 69120, GERMANY.
- Department of Radiation Oncology, University Hospital of Düsseldorf, Moorenstraße 5, Düsseldorf, NRW, 40225, GERMANY.
- Department of Radiation Oncology, University Hospital Heidelberg, Im Neuenheimer Feld 400, Heidelberg, 69120, GERMANY.
- Medical Physics Unit, Dipartimento di Diagnostica per Immagini e Radioterapia oncologica, Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Largo Agostino Gemelli 8, Rome, 00168, ITALY.
- Institute of Radiology, Università Cattolica del Sacro Cuore, Largo Agostino Gemelli 8, Rome, 00168, ITALY.
- Radiation therapy unit, Dipartimento di Diagnostica per Immagini e Radioterapia oncologica, Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Largo Agostino Gemelli 8, Rome, 00168, ITALY.
- Department of Radiation Oncology, University Hospital Zurich, Rämistrasse 100, Zurich, 8091, SWITZERLAND.
- Department of Radiation Oncology, University Hospital of Munich, Marchioninistr. 15, Munich, 81377, GERMANY.
- Department of Medical Physics, Ludwig-Maximilians-Universität München, Am Coulombwall 1, Munich, 85748, GERMANY.
Abstract
Magnetic resonance imaging (MRI) guided radiotherapy (MRgRT) requires the delineation of gross tumor volumes (GTV) in daily MRI from MRI-linacs. Specialized models have been developed for this task for certain tumors. This study investigated an alternative, using promptable foundation models.
Approach. Promptable foundation models were prompted with six different sparse geometric prompt types (points, boxes, 2D masks) to produce GTV segmentation masks, including Segment-anything 2 (SAM2), SAM2 fine-tuned for medical imaging (MedSAM2), and nnInteractive, an nnUnet-based promptable model for medical imaging. A diverse multi-institutional dataset of clinical GTV masks from the abdomen, lung, liver, pancreas, and pelvis sites on MRI scans from MRI-linacs was used to evaluate model outputs using various metrics, including the Dice similarity coefficient (DSC).
Main results. The models produced segmentation masks comparable or superior to those from domain-specific models with median DSCs of up to 0.85 (nnInteractive-mask3 prompt). Prompts with more spatial information yielded better results with lower variance, with the effect reduced for nnInteractive and MedSAM2. These produced overall better results (median DSC over all prompt types 0.75 for nnInteractive, 0.70 for MedSAM2, 0.54 for SAM2).
Significance. This investigation showed that promptable foundation models can in principle be used for GTV segmentation in MRI across multiple tumor types, although more research is necessary to reduce the variance and improve model performance.