Multimodal deep feature fusion with transformer for brain tumor classification from magnetic resonance imaging.
Authors
Affiliations (7)
Affiliations (7)
- Department of Computer Science and Engineering, Jain Deemed To Be University, Bangalore, Karnataka, 562112, India.
- School of Computing, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Avadi, Chennai, India.
- Department of Computer Science and Engineering, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Avadi, Chennai, 600062, India.
- School of Computer Science and Applications, S-VYASA (Deemed to be University), Bengaluru, Karnataka, 560059, India.
- School of Computer Applications, Kalinga Institute of Industrial Technology (KIIT) Deemed to be University, Bhubaneswar, India.
- School of Computer Science & Engineering, VIT-AP University, Amaravati, Andhra Pradesh, India. [email protected].
- Department of AI&DS, ICFAITECH, Faculty of Science and Technology), IFHE, Hyderabad, India.
Abstract
Brain tumors (BTs) arise due to abnormal cell growth, which has a high mortality rate globally. Millions of lives can be saved through the timely identification of BT. Precise identification and segmentation of BTs are essential to enhance the precision of analysis and the efficiency of therapeutic strategies. Magnetic resonance imaging (MRI) is a broadly utilized analytical tool. Furthermore, deep learning (DL) has recently shown efficiency in addressing several computer vision tasks. Several DL-driven methods are implemented for BT segmentation and attained impressive outcomes. This study presents a Multimodal Deep Feature Fusion Framework for Automated Brain Tumor Detection and Segmentation (MDFF-ABTDS) model. This objective is to develop a multimodal DL that integrates feature fusion and transformer networks for the precise detection and segmentation of BTs from medical images. Initially, image pre-processing is performed using Contrast Limited Adaptive Histogram Equalization (CLAHE) and image normalization. Feature extraction is carried out through fusion models such as CapsNet, ResNet-50, and AlexNet. These extracted features are then passed to a bi-directional convolutional long short-term memory combined with transformer (TBConvL-Net) models to classify tumors and non-tumors effectively. Finally, the tumor is classified to identify its location using the nnUNet model for a precise segmentation process. A series of experimental analyses of the MDFF-ABTDS method portrayed a superior accuracy value of 98.91% over existing models under the BT MRI dataset.