A multimodal evidence-driven framework for clinical decision support in cognitive impairment.
Authors
Affiliations (15)
Affiliations (15)
- Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China.
- Guangzhou National Laboratory, Guangzhou, China.
- School of Life Sciences, Peking University, Beijing, China.
- School of Life Sciences and Technology, ShanghaiTech University, Shanghai, China.
- GMU-GIBH Joint School of Life Sciences, Guangdong Provincial Key Laboratory of Protein Modification and Disease, The Guangdong-Hong Kong-Macao Joint Laboratory for Cell Fate Regulation and Diseases, Guangzhou Medical University, Guangzhou, China.
- Bioland Laboratory, Guangzhou, China. [email protected].
- Feihe Reseach Institute, Heilongjiang Feihe Dairy Co., Ltd., Chaoyang District, Beijing, China. [email protected].
- Guangzhou National Laboratory, Guangzhou, China. [email protected].
- GMU-GIBH Joint School of Life Sciences, Guangdong Provincial Key Laboratory of Protein Modification and Disease, The Guangdong-Hong Kong-Macao Joint Laboratory for Cell Fate Regulation and Diseases, Guangzhou Medical University, Guangzhou, China. [email protected].
- Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China. [email protected].
- Guangzhou National Laboratory, Guangzhou, China. [email protected].
- GMU-GIBH Joint School of Life Sciences, Guangdong Provincial Key Laboratory of Protein Modification and Disease, The Guangdong-Hong Kong-Macao Joint Laboratory for Cell Fate Regulation and Diseases, Guangzhou Medical University, Guangzhou, China. [email protected].
- Bioland Laboratory, Guangzhou, China. [email protected].
- School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China. [email protected].
- Shanghai Institute of Nutrition and Health, Chinese Academy of Sciences, Shanghai, China. [email protected].
Abstract
Deep learning approaches for cognitive impairment diagnosis have shown considerable promise, but their clinical translation remains limited by poor interpretability and weak linkage between model outputs and established medical evidence. Here we developed the Multimodal Evidence-Driven Reasoning Framework (MEDRF), which integrates a Multimodal Hierarchical Cascade (mHC) classifier with a retrieval-augmented large language model (RAG-LLM) for evidence-guided reasoning. MEDRF leverages routinely collected non-invasive data from clinical profiles and structural MRI to identify cognitive impairment stages and etiologies. Across 15 diagnostic labels, mHC outperformed flat multimodal baselines, supporting hierarchical diagnostic modeling. When the mHC was evaluated under progressive feature masking, performance declined with increasing missingness, whereas RAG-LLM correction mitigated this effect, especially under severe sparsity. In external validation on a heterogeneous cohort with primary labels, integrating RAG-LLM with the mHC improved all evaluation metrics, increasing overall accuracy from 0.706 ± 0.038 to 0.753 ± 0.032. The RAG module resolves ambiguous predictions by retrieving analogous cases and supporting textual evidence, enabling multi-hop reasoning across conflicting clinical cues. Physician review further indicated favorable quality and perceived usefulness of the generated reports. By synthesizing hierarchical prediction with evidence-grounded reasoning, MEDRF provides a robust interpretable framework for decision support, particularly in incomplete or diagnostically ambiguous presentations.