Clinical pathways matter for multimodal deep learning in early Alzheimer's disease detection.
Authors
Affiliations (5)
Affiliations (5)
- Department of Computer Science and Electrical Engineering, University of Stavanger, Kjell Arholms Hus 41, 4021, Stavanger, Norway.
- Department of Computer Science, University of Putra Malaysia, Jalan Universiti 1, 43400, Selangor, Malaysia.
- SMIL, Department of Radiology, Stavanger University Hospital, Gerd- Ragna Bloch Thorsens Gate 8, 4011, Stavanger, Norway.
- Centre for Age-Related Medicine (SESAM), Stavanger University Hospital, Gerd-Ragna Bloch Thorsens gate 8, Stavanger, 4011, Norway.
- Department of Computer Science and Electrical Engineering, University of Stavanger, Kjell Arholms Hus 41, 4021, Stavanger, Norway. [email protected].
Abstract
Identifying individuals at risk of Alzheimer's disease (AD), particularly in the preclinical and early stages, remains challenging. Although deep learning approaches based on structural MRI show promise as a non-invasive biomarker, existing multimodal models require task-specific training and depend on biomarkers that are not routinely available in clinical practice. Here, we propose a zero-shot multimodal feature extraction framework based on SigLIP that combines structural MRI embeddings with text embeddings of routinely collected clinical variables for early AD risk stratification in individuals at preclinical or mild cognitive impairment (MCI) stages. We evaluated the approach in 416 individuals from the ADNI cohort (age: 72.73 ± 6.7). SigLIP was used without fine-tuning to extract MRI and clinical text embeddings, which were combined into multimodal representations for individual-level AD risk prediction within 4 years. We further compared the model performance in a single-visit and two-visit settings to assess the value of longitudinal information and framework scalability. In the 1-visit setting, combining MRI embeddings with MMSE, age, and sex achieved an AUC of 0.91 ± 0.02, showing higher performance than the CSF Aβ42-based model (AUC 0.73 ± 0.08) and MMSE-based model (AUC 0.85 ± 0.22). In the 2-visit setting, performance was maintained or improved, supporting the scalability of the approach to longitudinal data. These findings suggest that multimodal fusion of SigLIP-derived MRI features and routinely collected clinical variables may provide a practical and scalable strategy for early AD risk progression prediction without task-specific training.