Preoperative identification of tumor deposits in rectal cancer using a transformer-based multimodal fusion model: a multicenter retrospective study.
Authors
Affiliations (4)
Affiliations (4)
- Department of Magnetic Resonance Imaging Diagnostic, The Second Affiliated Hospital of Harbin Medical University, No. 246, Xuefu Road, Nangang, Harbin, 150086, China.
- Department of Magnetic Resonance Imaging Diagnostic, The Fifth Affiliated Hospital of Harbin Medical University, No. 241, Jianshe Road, Development District, Daqing, Heilongjiang Province, 163316, China.
- Medical Imaging Center, Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University, No. 79, Kangning Road, Xiangzhou District, Zhuhai, Guangdong Province, 519000, China. [email protected].
- Department of Magnetic Resonance Imaging Diagnostic, The Second Affiliated Hospital of Harbin Medical University, No. 246, Xuefu Road, Nangang, Harbin, 150086, China. [email protected].
Abstract
To develop and validate a transformer-based deep learning-radiomics model for the non-invasive preoperative discrimination of tumor deposits (TDs) in rectal cancer by integrating multi-sequence MRI features and clinical risk factors. This multicenter retrospective study enrolled 684 patients with pathologically confirmed rectal adenocarcinoma from three hospitals. The cohort distribution was as follows: 425 patients from Center 1 were randomly split in a 7:3 ratio into an internal training set and an internal validation set; Center 2 contributed 154 patients; and Center 3 provided 105 patients. Radiomics features (including novel topological and Hessian matrix features) and deep learning features based on DenseNet-101 were extracted from T2WI and DWI sequences, while key clinical features were screened. All features were then subjected to standardization and dimensionality reduction before being input into a self-attention-based Transformer encoder for deep fusion.Model performance was evaluated using receiver operating characteristic (ROC) analysis, decision curve analysis (DCA), calibration curves, and the net reclassification index (NRI). The transformer-based fusion model demonstrated superior performance, achieving AUCs of 0.974, 0.742, 0.746, and 0.752 in the training, internal validation, external validation cohort 1, and external validation cohort 2, respectively. It showed optimal accuracy, stability, and the highest net clinical benefit across a wide threshold probability range. The NRI indicated a significant improvement (62.6%) over the traditional deep neural network fusion model. The MRI-based transformer multimodal fusion model enhances the capability to preoperatively identify tumor deposits in rectal cancer with high accuracy. By providing a non-invasive and reliable tool for risk stratification, this approach holds the potential to optimize individualized treatment planning and improve patient outcomes.