Multi-modal deep learning model for predicting recurrence of moderately severe and severe acute pancreatitis.
Authors
Affiliations (7)
Affiliations (7)
- State Key Laboratory of Complex, Severe, and Rare Diseases, Biomedical Engineering Facility of National Infrastructures for Translational Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China.
- Department of Internal Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China.
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China.
- Department of Nuclear Medicine, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China.
- Department of Radiology, Peking Union Medical College Hospital, Beijing 100730, China. Electronic address: [email protected].
- State Key Laboratory of Complex, Severe, and Rare Diseases, Biomedical Engineering Facility of National Infrastructures for Translational Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China. Electronic address: [email protected].
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China; Department of Gastroenterology, Tibet Autonomous Region People's Hospital, Lhasa 850005, China; Clinical Epidemiology Unit, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China. Electronic address: [email protected].
Abstract
To overcome the limitations of single-modality predictors by developing and validating a multimodal model (APNet) that integrates clinical factors and contrast-enhanced CT features to predict recurrence of moderate-to-severe acute pancreatitis (MSAP/SAP). We retrospectively collected clinical data and enhanced CT images from a total of 235 patients with moderate-to-severe AP. To rigorously evaluate model generalizability, the dataset was divided into two distinct cohorts: a Development Cohort (N = 184) for model training and internal cross-validation, and an Independent Validation Cohort (N = 51) for performance evaluation. Clinical machine learning models were first developed, followed by APNet, a multimodal deep learning model integrating ResNet- and ViT-extracted CT features with clinical risk factors through multiscale fusion. Among single-modality approaches, the LightGBM model using clinical data achieved an AUC of 0.711, while image-based deep learning with ResNet50 reached an AUC of 0.815. The proposed multimodal fusion model, APNet, showed the best predictive performance, achieving an AUC of 0.840 on the independent test set, with corresponding accuracy, precision, recall, and F1 score of 82.35%, 66.67%, 80.00%, and 72.73%. Overall, APNet consistently outperformed all single-modality models, highlighting the complementary value of combining imaging features with clinical risk factors. APNet effectively integrates clinical and imaging data, significantly improving prediction of recurrence in MSAP/SAP patients. This multimodal tool can help identify high-risk MSAP and SAP patients early, supporting targeted interventions and better long-term outcomes.