Multimodal deep learning framework for recurrence risk stratification in soft tissue sarcoma: a multicenter study.
Authors
Affiliations (7)
Affiliations (7)
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China.
- Department of Research Collaboration, R&D center, Hangzhou Deepwise & League of PHD Technology Co. Ltd., Hangzhou, Zhejiang, China.
- School of Engineering Medicine, Beihang University, Beijing, China.
- Department of Pathology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China.
- Department of Pathology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, Shandong, China.
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, Shandong, China. [email protected].
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China. [email protected].
Abstract
Accurate prediction of recurrence risk is essential to devise effective and personalized treatment strategies for patients with soft tissue sarcoma (STS). This study aimed to develop and validate a multimodal deep learning framework that integrates clinical features, preoperative MR images, and hematoxylin and eosin-stained whole slide images (WSIs) to predict recurrence in patients with STS. A total of 323 patients with STS were retrospectively enrolled from two hospitals, serving as development and validation sets, respectively. The ShuffleNetV2 network was utilized to develop patch-level and WSI-level signatures. A convolutional neural network fusing the channel and spatial attention mechanisms was used to develop a radiology signature. The combined model was built by integrating clinical features, radiology signature score, and WSI-level signature score with Cox regression analysis. The combined model demonstrated superior performance in the validation set, achieving a C-index of 0.857 and a time-dependent area under the curve of 0.959. Class activation maps facilitated the monitoring of suspected regions to inform recurrence decisions. The recurrence-free survival times of the low- and high-risk cohorts were statistically different (p < 0.05). The proposed multimodal framework offers satisfactory accuracy for predicting recurrence risk in patients with STS and could guide the choice of treatment modality.