Development and validation of a novel multimodal deep learning model integrating histopathology, radiology, and clinical data to predict primary non-response to infliximab in patients with crohn's disease.
Authors
Affiliations (5)
Affiliations (5)
- Department of Gastroenterology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China.
- Medical AI Lab, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen, Guangdong, China.
- Department of Pathology, The First Affiliated Hospital, Sun Yat-Sen University, Guangzhou, 58, China.
- Department of Radiology, The First Affiliated Hospital, Sun Yat-Sen University, Guangzhou, 58, China.
- Department of Gastroenterology, Chongqing Key Laboratory of Digestive Malignancies, Daping Hospital, Army Medical University (Third Military Medical University), Chongqing, China.
Abstract
Crohn's disease (CD) is a chronic inflammatory condition of the gastrointestinal tract. While infliximab (IFX) offers significant benefits, 10-30% of patients remain non-responders initially. This study employs artificial intelligence with multimodal integration to improve treatment response prediction and advance precision medicine. We conducted a retrospective analysis of clinical data from patients with CD. The endpoint event was defined as primary non-response within 14 weeks of treatment. The multimodal dataset included laboratory indices, Computed Tomography Enterography (CTE), and endoscopic histopathology based on whole-slide biopsy images. A TabNet model, originally designed for tabular data and here applied to clinical and laboratory features, was developed using a multi-instance learning framework to incorporate this multimodal information for predicting primary non-response to IFX. Finally, the multimodal model was validated in an independent external test cohort. The study included 188 patients, with 93 in the internal training set, 38 in the internal validation set, and 57 in the test set from an independent external cohort. The model utilizing pathological features achieved an area under the receiver operating characteristic (AUC) of 0.789 in internal validation. When combining pathological and radiological features, the AUC was 0.844. The optimal multimodal model integrating histology, radiology, and clinical features achieved an AUC of 0.852 in internal validation set and 0.858 in external test set. The study developed a multimodal deep learning model accurately predicting IFX primary non-response, offering a tool to guide individualized therapy and improve Crohn's disease outcomes.