Multi-Scale Graph-Based Cross-Attention Transformer for Whole Slide Image Classification

As Whole Slide Images(WSI) are characterized by their extremely high resolution and lack of pixel-level annotation, Multiple Instance Learning(MIL) has become an effective tool for addressing WSI classification challenge. However, current MIL methods typically rely on the independent and identically distributed assumption, neglecting correlation among different instances and only analyze WSI at a single scale, failing to effectively utilize the complementarity between multi-scale features. To enhance classification performance and obtain more accurate diagnostic results, we propose a MIL model based on multi-scale Graph-Transformer framework. It utilizes graph convolutional network to capture spatial, semantic, and scale correlations among patches, and performs graph pooling to select discriminative patches in order to reduce computational costs. Additionally, we introduce a multi-scale feature fusion module aimed at integrating features across different scales. The experimental results on the TCGA-NSCLC and TCGA-RCC datasets show that our method achieves consistent high performance for different cancer types, obtaining accuracy of 0.9281 and 0.9508 on the above two datasets, respectively. Compared with other comparison methods, our method has surperior classification performance.