Multimodal fusion of ultrasound images using HXM net for breast cancer diagnosis.
Authors
Affiliations (5)
Affiliations (5)
- Department of Computer Science and Engineering, Alliance College of Engineering and Design, Alliance University, Bengaluru, India.
- Data Science Department, Alliance University, Bengaluru, India.
- Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Guntur, Andhra Pradesh, India.
- Department of Mechanical Engineering Department, Madanapalle Institute of Technology & Science, Madanapalle, 517325, India.
- School of Mechanical Engineering, Vellore Institute of Technology Chennai, Chennai, Tamil Nadu, 600127, India. [email protected].
Abstract
Breast cancer is a major global health issue in women, where diagnosis at an early stage is decisive for enhancing the effectiveness of treatment and survival. Despite the advances in imaging using medical technologies, maintaining uniformly good diagnostic accuracy is challenging due to the difficulty of lesion characterization and limitations of single-modality imaging. This work presents HXM-Net, a deep learning model specifically tailored to improve breast cancer detection through the synergistic benefit of multi-modal ultrasound imaging. HXM-Net combines Convolutional Neural Networks (CNNs) for spatial feature extraction and Transformer-based fusion for optimal concatenation of information from the B-mode and Doppler ultrasound images. The dual modality-based method captures morphological and vascular features of breast lesions and produces a more informative and discriminative feature representation. The model embeds multi-scale feature learning and data augmentation to guarantee generalizability to different lesion types and patient populations. In the tests conducted on a class-balanced breast ultrasound database, the HXM-Net achieved accuracies of 94.20%, sensitivity (recall), 92.80%, specificity, 95.70%, an F1 score of 91.00%, and AUC-ROC of 0.97, thereby establishing the superiority of HXM-Net over conventional models like ResNet-50 and U-Net, especially in distinguishing benign tumours from malignant ones. The strong diagnostic ability and versatility of the model make it a good contender for integration with clinical decision support systems that can facilitate more assured diagnostics and enhanced patient care.