Marrying Global–Local Spatial Context for Image Patches in Computer-Aided Assessment

Computer-aided assessment using whole slide images (WSIs) is one of the critical steps in clinical procedures. How do doctors recognize cancer in a WSI? A quick answer is that they consider the spatial structure of a WSI rather than only considering single patches. We argue that two clues are essential for computer-aided deep learning: 1) global spatial context and 2) local semantic information. This is because local, semi-local, and global tissue observing are the principal assessment means of pathologists, perfectly corresponding with both clues. However, most existing methods only consider local spatial information learning within each patch rather than developing an effective local-to-global reaction, leading to an incapable of capturing robust and enriched representation. Toward a new area for computer-aided assessment, we propose novel neural networks to learn the global–local spatial context in WSIs, called GLSCL. The GLSCL is among the first trials that understand both clues for WSI understanding. Furthermore, the proposed novel operators enable the GLSCL to learn spatial semantic representation sufficiently. We evaluate the GLSCL using renal cell carcinoma (RCC) samples with synthetic ambiguity collected from the public benchmark and clinical procedures. Enhanced by global and local spatial information, the GLSCL achieves state-of-the-art performance, including classification accuracy, survival prediction index, and cancer tissue attention rate.