Development and validation of a multimodal AI-agent system for prognosis analysis of bladder urothelial carcinoma.
Authors
Affiliations (12)
Affiliations (12)
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China.
- Department of Pathology, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China.
- Department of Pathology, Yongchuan Hospital of Chongqing Medical University, Chongqing, China.
- Department of Pathology, College of Basic Medicine, Chongqing Medical University, Chongqing, China.
- Department of Clinical Pathololgy Laboratory of Pathology Diagnostic Center, Chongqing Medical University, Chongqing, China.
- Molecular Medicine Diagnostic and Testing Center, Chongqing Medical University, Chongqing, China.
- Department of Radiology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China.
- College of Artificial Intelligence Medicine, Chongqing Medical University, Chongqing, China. [email protected].
- College of Biomedical Engineering, Chongqing Medical University, Chongqing, China. [email protected].
- Department of Urology, The Third Affiliated Hospital of Chongqing Medical University, Chongqing, China. [email protected].
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China. [email protected].
- Department of Urology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China. [email protected].
Abstract
Precise survival risk stratification for bladder urothelial carcinoma (BUC) remains a clinical challenge. We developed and validated a multimodal AI agent that integrates textual, radiographic, and pathological data from 1185 patients across four medical centers to predict survival risk. The agent employs LLMs to standardize pathology reports, interactive deep learning networks for precise CT image segmentation, and extracts features from CT scans and whole slide images using CTVisionNet and MacroVisionNet. The multimodal fusion framework, MATCH-Net, integrates these features with microscopic pathology information and clinical text embeddings using a multi-head attention mechanism to generate a comprehensive prognostic score. In multi-center validation, MATCH-Net demonstrated robust performance (C-index ranging from 0.836 to 0.874) and effectively stratified patients into high- and low-risk groups, identifying potential candidates responsive to adjuvant chemotherapy. Furthermore, the framework enabled the quantification of novel, interpretable prognostic biomarkers and provides a reliable and clinically applicable solution for personalized BUC prognosis.