A pre-trained foundation model framework for multiplanar MRI classification of extramural vascular invasion and mesorectal fascia invasion in rectal cancer.
Authors
Affiliations (9)
Affiliations (9)
- The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands.
- Research & Frontiers in AI Department, Quantitative Imaging Biomarkers in Medicine, Quibim SL, Valencia, Spain.
- Biomedical Imaging Research Group, La Fe Health Research Institute, Valencia, Spain.
- Radiology Unit, Department of Surgical and Medical Sciences and Translational Medicine, Sapienza University of Rome, Sant'Andrea Hospital, Rome, Italy.
- Laboratoire HIFIH, Université d'Angers, SFR ICAT 4208, Angers, France.
- Department of Radiology, CHU Angers, Angers, France.
- Medical Imaging Department, La Fe University and Polytechnic Hospital, Valencia, Spain.
- The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, The Netherlands. [email protected].
- Department of Radiology and Nuclear Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands. [email protected].
Abstract
Accurate MRI-based identification of extramural vascular invasion (EVI) and mesorectal fascia invasion (MFI) is crucial for risk-stratified rectal cancer treatment. However, subjective visual assessment and inter-institutional variability limit diagnostic consistency. This study developed and evaluated a multi-center, foundation model-driven framework that automatically classifies EVI and MFI on axial and sagittal MRI. A total of 331 pre-treatment rectal cancer T2-weighted MRI scans from three European hospitals were retrospectively recruited. A self-supervised frequency domain harmonization strategy was applied to reduce scanner variability. Three classifiers, SeResNet, the universal biomedical pretrained model (UMedPT) with a multilayer perceptron head, and a logistic-regression variant using frozen UMedPT features (UMedPT_LR), were trained (n = 265) and tested (n = 66). Gradient-weighted class activation mapping (Grad-CAM) visualized model predictions. UMedPT_LR achieved the best EVI performance with multiplanar fusion (AUC = 0.82, test set). For MFI, UMedPT trained on axial harmonized images yielded the highest performance (AUC = 0.77). Both tasks outperformed the CHAIMELEON 2024 benchmark (EVI: 0.82 vs 0.74; MFI: 0.77 vs 0.75). Harmonization enhanced MFI classification, and multiplanar fusion further boosted EVI performance. Grad-CAM confirmed biologically plausible attention on peritumoral regions (EVI) and mesorectal fascia margins (MFI). The proposed foundation model-driven framework, leveraging frequency domain harmonization and multiplanar fusion, achieves state-of-the-art performance for automated EVI and MFI classification on MRI, demonstrating strong generalizability across multiple centers. Addressing inter-center inconsistencies in rectal cancer MRI, a multiplanar foundation model with cross-scanner harmonization significantly improves the detection of EVI and MFI, potentially standardizing staging and guiding therapy. Among the first studies to investigate automated classification of both EVI and MFI using axial and sagittal T2-weighted MRI. Foundation model-derived features outperform conventional convolutional neural networks (CNNs) for EVI and MFI classification. Frequency domain harmonization and multiplanar fusion selectively enhance diagnostic performance. Automated prediction of EVI and MFI may support more consistent staging and clinical decision-making across institutions.