Modality-AGnostic image Cascade (MAGIC) for multi-modality cardiac substructure segmentation.
Authors
Affiliations (12)
Affiliations (12)
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA; Department of Human Oncology, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Computer Science, Wayne State University, Detroit, MI, USA. Electronic address: [email protected].
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA; Department of Human Oncology, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA; Department of Human Oncology, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Radiation Oncology, Henry Ford Cancer Institute, Detroit, MI, USA; Alexandria Department of Clinical Oncology, Faculty of Medicine, Alexandria University, Alexandria, Egypt. Electronic address: [email protected].
- Department of Radiation Oncology, The Ohio State University, Columbus, OH, USA. Electronic address: [email protected].
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Radiology, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Statistics, University of Wisconsin-Madison, Madison, WI, USA; Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
- Department of Computer Science, Wayne State University, Detroit, MI, USA. Electronic address: [email protected].
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA; Department of Human Oncology, University of Wisconsin-Madison, Madison, WI, USA. Electronic address: [email protected].
Abstract
Cardiac substructure delineation is emerging in treatment planning to minimize the risk of radiation-induced heart disease. Deep learning offers efficient methods to reduce contouring burden but currently lacks generalizability across different modalities and overlapping structures. This work introduces and validates a Modality-AGnostic Image Cascade (MAGIC) deep-learning pipeline for comprehensive and multi-modal cardiac substructure segmentation. MAGIC is implemented through replicated encoding and decoding branches of an nnU-Net backbone to handle multi-modality inputs and overlapping labels. First benchmarked on the multi-modality whole-heart segmentation (MMWHS) dataset including cardiac CT-angiography (CCTA) and MR modalities, twenty cardiac substructures (heart, chambers, great vessels (GVs), valves, coronary arteries (CAs), and conduction nodes) from clinical simulation CT (Sim-CT), low-field MR-Linac, and cardiac CT-angiography (CCTA) modalities were delineated to train semi-supervised (n = 151), validate (n = 15), and test (n = 30) MAGIC. For comparison, fourteen single-modality comparison models (two MMWHS modalities and four subgroups across three clinical modalities) were trained. Methods were evaluated for efficiency and against reference contours through the Dice similarity coefficient (DSC) and two-tailed Wilcoxon Signed-Rank test (p < 0.05). Average MMWHS DSC scores across CCTA and MR inputs were 0.88 ± 0.08 and 0.87 ± 0.04 respectively with significant improvement (p < 0.05) over unimodal baselines. Average 20-structure DSC scores were 0.75 ± 0.16 (heart, 0.96 ± 0.01; chambers, 0.89 ± 0.05; GVs, 0.81 ± 0.09; CAs, 0.60 ± 0.13; valves, 0.70 ± 0.18; nodes, 0.66 ± 0.12) for Sim-CT, 0.68 ± 0.21 (heart, 0.94 ± 0.01; chambers, 0.87 ± 0.05; GVs, 0.72 ± 0.18; CAs, 0.50 ± 0.18; valves, 0.62 ± 0.16; nodes, 0.52 ± 0.16) for MR-Linac, and 0.80 ± 0.16 (heart, 0.95 ± 0.01; chambers, 0.93 ± 0.04; GVs, 0.84 ± 0.06; CAs, 0.77 ± 0.12; valves, 0.68 ± 0.23; nodes, 0.72 ± 0.11) for CCTA. Furthermore, >80 % and > 70 % reductions in training time and parameters were achieved, respectively. MAGIC offers an efficient, lightweight solution capable of segmenting multiple image modalities and overlapping structures in a single model without compromising segmentation accuracy.