A Hybrid Deep Learning Approach for Brain Tumor Recognition Using CNNs, Transformers, and Multi-Modal MRI Data

Brain tumor detection and analysis are critical for early diagnosis and effective treatment planning. In this proposed method a hybrid deep learning representation that integrates EfficientNetV2, Vision Transformer (ViT), and DeepMed3D for comprehensive brain tumor recognition. The proposed model consists of three key tasks: tumor classification, tumor segmentation, and multi-modal MRI analysis. For tumor classification, EfficientNetV2 is employed for feature extraction, while ViT enhances global attention learning, achieving high accuracy (>98%) in distinguishing tumors from non-tumors. Tumor segmentation is performed using Swin Transformer-based UNETR, which effectively captures both local and global spatial features, yielding a Dice Score of 90-95%. For multi-modal MRI analysis, DeepMed3D processes T1, T2, and FLAIR sequences, leveraging self-attention mechanisms to enhance feature fusion, achieving 98%+ accuracy in multi-class tumor classification (benign, malignant, normal). Massive experimentations on benchmark MRI datasets prove that proposed method betters existing advanced models in accuracy, precision, recall, and segmentation quality. The integration of CNNs and Transformers enables better feature representation, while multi-modal fusion enhances robustness. The results confirm that the proposed approach significantly improves brain tumor detection, segmentation, and classification, making it a promising tool for medical applications.