Vision Transformer for Brain Tumor Detection Using Hyperspectral Images With Reduced Spectral Bands

Brain tumors, particularly gliomas, pose a significant clinical challenge with rising incidence rates and high mortality. Artificial intelligence combined with Hyperspectral Imaging (HSI) offers promising tools to improve surgical precision and patients’ outcomes. HSI offers unique advantages, including non-invasiveness and detailed spectral data, for enhanced tumor tissue differentiation. This study tailored the Vision Transformer (ViT) with techniques from remote sensing to segment nineteen spectral images of low- and high-grade gliomas with limited spectral bands. The choice of the ViT was motivated by its attention mechanism, enabling fine-grained distinction of subtle details. Segmentation focused on four classes: healthy tissue, tumor tissue, blood vessels and dura mater. A careful hyperparameter optimization was performed, resulting in the selection of two models based on a defined quality index, which were evaluated using three experimental methodologies, achieving up to $98.24\pm 2.50$ % average Overall Accuracy (OACC) and $99.61\pm 0.66$ % average Area Under the Curve (AUC) in intra-patient classification. For inter-patient classification, the models achieved an average OACC up to $53.56\pm 24.91$ % and an average AUC score up to $79.27\pm 10.43$ %, highlighting areas of improvement. Comparable or improved performance was demonstrated versus other deep learning techniques applied to the same dataset, proving effectiveness with few spectral bands. Some results were lower than a similar application with more bands, but they also underscore the adaptability and potential of the ViT to handle challenging datasets. Insights from hyperparameter optimization shows the ViT’s promise as a robust tool for tumor identification, paving the way for integration into real-time clinical workflows and advancing precision medicine.