Prior knowledge-guided multimodal deep learning system for biomarker exploration and prognosis prediction of urothelial carcinoma.
Authors
Affiliations (13)
Affiliations (13)
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, PR China.
- Department of Pathology, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, PR China.
- Department of Urology, The Third Affiliated Hospital of Chongqing Medical University, Chongqing, PR China.
- Department of Breast Cancer Center, Chongqing Cancer Hospital, Chongqing, PR China.
- Department of Pathology, College of Basic Medicine, Chongqing Medical University, Chongqing, PR China.
- Laboratory of Pathology Diagnostic Center, Department of Clinical Pathology, Chongqing Medical University, Chongqing, PR China.
- Molecular Medicine Diagnostic and Testing Center, Chongqing Medical University, Chongqing, PR China.
- Department of Radiology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, PR China.
- College of Biomedical Engineering, Chongqing Medical University, Chongqing, PR China.
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, PR China. [email protected].
- Department of Pathology, Yongchuan Hospital of Chongqing Medical University, Chongqing, PR China. [email protected].
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, PR China. [email protected].
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, PR China. [email protected].
Abstract
Accurate survival prediction for urothelial carcinoma (UC) is critical but limited by a lack of AI systems that integrate multimodal data with prior knowledge. To address this gap, we developed a multimodal deep learning system that integrates histopathology, radiology, and structured pathology text. We incorporated prior knowledge to improve tumor segmentation and create knowledge-guided slide representations. CTContextNet and MacroContextNet were then employed for capturing radiological and macroscopic prognostic information. IM-NCTNet integrated multi-modality information for enhanced prediction. Across multi-center, large-scale, multi-cohort validation, the knowledge-guided prognostic system demonstrated higher performance over single-modality models, achieving C-index scores ranging from 0.809 to 0.867. The framework identified novel prognostic biomarkers related to infiltration patterns in muscle and renal parenchyma. High-risk Coloc_M, Coloc_R, and IMTS groups exhibited increased mortality risk, with hazard ratio values ranging from 2.47 to 16.38. The proposed AI framework offers a comprehensive and robust tool for UC prognosis, supporting refined patient management.