Deep Learning-Based Multi-Class Model in WSIs: Vision Transformer for Distinguishing Cancer, Tumor, Mitosis, Red Blood Cells, and Karyorrhexis

Neuroblastoma (NB) is a heterogeneous group of pNTs primarily affecting children under five years of age. NB exhibits significant molecular, histopathological, and clinical variability, necessitating distinct risk stratification and management strategies. This study investigates the application of deep learning techniques, specifically Vision Transformers (ViT) and ResNet50, for multi-class cancer classification using Whole Slide Images (WSIs) of NB tissues. We introduce a fine-tuned model with ViT and ResNet50 to differentiate multiple cellular classes, including tumor cells, mitotic figures, red blood cells, and karyorrhexis (indicative of necrotic or apoptotic tumor cells). Our primary aim is to distinguish the tumor mitosis-karyorrhexis index, a critical metric for NB prognosis. For this research, we compile and annotate a dataset from 25 patients sourced from three medical schools in the United States of America. We pre-processed a dataset comprising 14,960 cell images from 37 hematoxylin and eosin-stained histological images. Medical experts perform cellular-level annotations on WSIs using QuPath, outlining the spatial extent of interested cells through bounding boxes around cells to establish ground truth data for model training and evaluation. Our findings indicate that ViT achieves an accuracy of 90.16%, while ResNet-50 achieves an accuracy of 79%, underscoring ViT’s slightly higher accuracy in multi-class scenarios.CCS Concepts• Deep Learning → Convolutional Neural Networks (CNN), • CNN → Vision Transformer (ViT), • ViT → Cancer Classification.