BreastUS-Net: An Attention-Guided Dual-Branch Network with Feature Fusion for Fine-Grained Breast Tumor Classification in Ultrasound Imaging.
Authors
Abstract
Despite advances in deep learning (DL) and computer vision, breast cancer (BC) detection via ultra-sound remains challenging. Existing methods often focus on single tasks using complex pipelines and publicly available datasets, limiting clinical applicability. To address this, we propose BreastUS-Net-a novel architecture for hierarchical BC classification using diverse datasets. Our approach uses a dual-branch MobileNet architecture with fine-tuned and frozen layers to capture both task-specific and general features, eliminating manual feature extraction. These features are then fused to create a comprehensive representation, which is subsequently aggregated and refined. The aggregation step merges the outputs from both branches, while the refinement module reduces complexity, highlights relevant patterns, and mitigates overfitting to improve generalization. Additionally, we integrate a multihead self-attention (MHSA) block to highlight diagnostically significant regions in ultrasound images, enhancing both accuracy and robustness. Finally, the orthogonal softmax layer (OSL) boosts discriminative power by enforcing orthogonality among weight vectors, reducing parameter coadaptation and enabling more effective optimization. We used six diverse datasets from multiple centers, including: a large Zhejiang Cancer Hospital set (2,171 images), public BUSI dataset (780 images), external test sets from Yunnan Cancer Hospital (351 images) and Sir Run Run Shaw Hospitals (365 images), fibroadenoma (FA) vs. phyllodes tumor (PT) classification, and a PT grading dataset. We use explainable AI (XAI) techniques-Grad-CAM, SHAP, and saliency maps-to enhance trust in breast ultrasound predictions. Our model achieves state-of-the-art performance, with accuracies of 94.48% on a clinical dataset and 94.23% on the BUSI dataset, highlighting its potential to improve BC diagnosis and personalized treatment.