Integrating multimodal clinical data with a large model for prostate cancer diagnosis.
Authors
Affiliations (16)
Affiliations (16)
- Department of Urology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
- Shanghai Artificial Intelligence Laboratory, Shanghai, China.
- Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China.
- Department of Urology, The First Affiliated Hospital of Soochow University, Suzhou, China.
- Department of Urology, The Affiliated Hospital of Jiaxing University, Jiaxing, China.
- Department of Urology, Yancheng First Hospital, Affiliated Hospital of Nanjing University Medical School, The First people's Hospital of Yancheng, Jiangsu, China.
- Department of Urology, The First People's Hospital of Yulin (the Sixth Affiliated Hospital of Guangxi Medical University), Yulin, Guangxi, China.
- Department of Urology, Changhai Hospital, Second Military Medical University (Naval Medical University), Shanghai, China.
- Department of Urology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.
- Department of Radiation Oncology, Hospital Lüneburg, Lüneburg, Germany.
- Department of Urology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China. [email protected].
- Department of Urology, Changhai Hospital, Second Military Medical University (Naval Medical University), Shanghai, China. [email protected].
- Center for Genomic and Personalized Medicine, Guangxi key Laboratory for Genomic and Personalized Medicine, University Engineering Research Center of Digital Medicine and Healthcare, School of Life Science, Guangxi Medical University, Nanning, Guangxi, China. [email protected].
- Shanghai Artificial Intelligence Laboratory, Shanghai, China. [email protected].
- Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China. [email protected].
- Department of Urology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China. [email protected].
Abstract
Accurate prostate cancer (PCa) diagnosis remains difficult because of tumor heterogeneity and the challenge of integrating multimodal clinical information. We developed Prost-LM, a multimodal large language model that jointly embeds MRI-derived features, numerical PSA values, and free-text clinical reports into a unified semantic space to enable deep cross-modal reasoning. Trained and validated on a large multi-center cohort of 3940 patients, Prost-LM achieved strong diagnostic performance, with an internal validation AUC of 0.954 for distinguishing PCa from benign conditions, outperforming MRI-only models (AUC = 0.868, P < 0.001). For detecting clinically significant PCa (Gleason score ≥ 7), Prost-LM reached an AUC of 0.955. Additionally, the model provides interpretable diagnostic decisions to support clinical verification. These results suggest Prost-LM can improve automated PCa diagnosis and support precision oncology through multimodal AI.