Multimodal neuroimaging fusion with hierarchical structure-function coupling for autism spectrum disorder diagnosis.
Authors
Affiliations (4)
Affiliations (4)
- School of Communication and Electronic Engineering, Shandong Normal University, Jinan, China. Electronic address: [email protected].
- School of Communication and Electronic Engineering, Shandong Normal University, Jinan, China.
- Department of Biophysics and Radiation Biology, Semmelweis University, Budapest, Hungary.
- School of Information Science and Engineering, Shandong Normal University, Jinan, China. Electronic address: [email protected].
Abstract
Multimodal neuroimaging fusion has gained considerable attention in diagnosing autism spectrum disorder (ASD) due to its ability to integrate complementary information across different modalities. However, most existing approaches rely on simple feature concatenation or late fusion strategies without achieving effective cross-modal coupling, failing to capture the intrinsic relationships between functional and structural information. To address these limitations, this study develops the deep multimodal dual-level coupling (DMDC) framework integrating brain functional and structural information to improve model discrimination capability and interpretability. Specifically, the pyramid-inverted graph-attention network is first proposed to dynamically update graph topological structures with the Top-k node filtering algorithm for function-structure network coupling. Second, the skeleton-based white matter projection method maps functional magnetic resonance imaging (fMRI) signals onto diffusion tensor imaging (DTI) derived white matter skeletons, followed by the multi-layered hierarchical convolutional network for representation extraction. Finally, the weighted feature integration mechanism combines both components, and the neural network with cross-entropy loss optimization is employed for ASD identification. Extensive experiments show DMDC outperforms state-of-the-art methods. Key discriminative regions identified include the hippocampus, anterior cingulate cortex, amygdala, and frontal gyri. Both hyperconnectivity and hypoconnectivity were observed in ASD, especially in prefrontal cortex, amygdala, and hippocampus, which were critical for social and emotional processing. The proposed framework demonstrates robust diagnostic performance and provides reliable biomarkers for neurological assessment. The identified regions and connectivity patterns offer insights into ASD neural mechanisms and potential diagnostic biomarkers.