Trifocal Transformer: Connection-Mask-Residual Focused Attention Network for Brain Disease Diagnosis.
Authors
Abstract
Functional magnetic resonance imaging (fMRI) is a non-invasive neuroimaging technique that allows the observation of brain functional connectivity patterns. Attention-based diagnostic models like Transformers have been widely applied in fMRI data for brain disease diagnosis. However, the global attention mechanism of the Transformer faces challenges in adaptively identifying and focusing on significant brain regions and connections relevant to disease diagnosis while reducing attention to non-relevant regions and connections in fMRI data, as well as the degradation problem of the attention mechanism, thereby limiting the improvement in diagnostic accuracy. To address these problems, we propose a connection-mask-residual focused attention network (Trifocal Transformer) based on fMRI data for brain disease diagnosis. In the Trifocal Transformer, a Connection Focus Module is developed to simulate brain functional connectivity, thereby enhancing the attention mechanism's ability to focus on significant regions and connections relevant to disease diagnosis more effectively than standard self-attention mechanisms. To mitigate the potential negative impact of non-focused regions in the attention map, a learnable Mask Focus Module is designed to adaptively reduce attention to non-relevant regions and connections, addressing a limitation in conventional Transformer-based models. To address the degradation of the attention mechanism's focusing ability, we establish Residual Focus Connections between the attention maps, which reinforce the focusing effect across layers and ensure stable attention to significant features. Comprehensive experimental results demonstrate that the Trifocal Transformer achieves superior diagnostic accuracies of 74.1% and 71.2% on ADHD-200 and ABIDE I datasets, respectively. Furthermore, our method reveals potentially disease-related regions of interest (ROIs), providing a new neuroimaging perspective for brain disease diagnosis and treatment. Code is available at https://github.com/Jiarui-Liang/Trifocal-Transformer.