Global-local feature fusion: a robust hybrid deep learning model for multiclass brain tumor classification with Grad-CAM++ interpretation.
Authors
Affiliations (4)
Affiliations (4)
- Department of Computer Science and Engineering, Bipin Tripathi Kumaon Institute of Technology, Dwarahat, Uttarakhand, India.
- School of Computer Applications and Technology, Galgotias University, Greater Noida, Uttar Pradesh, India.
- Center for Precision Health, McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States.
- Department of Health Information Management and Technology, College of Applied Medical Sciences, King Faisal University, Al-Ahsa, Saudi Arabia.
Abstract
Accurate classification of brain tumors in MRI scans is critical for effective clinical decision making, yet manual assessment is labor-intensive and subject to variability. Although deep learning models, particularly CNNs and Transformers, have shown potential, each architecture alone has limitations: CNNs excel at local feature extraction but lack global context awareness, while Transformers capture global relationships but may overlook fine-grained details. To develop a hybrid Transfer Learning-Transformer model that integrates CNN-driven local feature modeling with Transformer-based global reasoning to enhance multi-class brain tumor classification. The proposed workflow comprises three stages: (1) benchmarking standalone CNN (ResNet50, VGG19, ConvNeXtBase, EfficientNetV2B0) and Transformer models (ViT, Swin, DeiT, PoolFormer); (2) constructing two intra-family hybrids-H<sub>TL</sub> (ResNet50 + ConvNeXtBase) and H<sub>TF</sub> (PoolFormer + ViT); and (3) fusing them into a final hybrid model (H<sub>F</sub>). Two public MRI datasets (Figshare, Kaggle) were used. Models were trained, validated, and tested on the Kaggle dataset, while the Figshare dataset was used exclusively for external validation. Performance was assessed using accuracy, precision, recall, F1-score, AUC, Friedman's aligned-rank test, Kendall's W, Holm <i>post hoc</i> analysis, TOPSIS-based ranking, calibration analysis (Brier score), and Grad-CAM++ for interpretability. The H<sub>F</sub> model achieved near-perfect classification (∼99.4% on Kaggle and up to 100% on Figshare), significantly outperforming all baseline and single-architecture models. Statistical and multi-criteria analyses consistently ranked H<sub>F</sub> as the top-performing model, while calibration results confirmed reliable probability estimation. Grad-CAM++ further indicated tumor-focused decision-making. The proposed hybrid model delivers high accuracy, strong generalizability, and reliable, interpretable predictions across MRI datasets, positioning it as a promising solution for AI-assisted brain tumor diagnosis.