A Spatiotemporal Cross-Recognition Network with Robust Generalization for Lesion Classification in Different Medical Images

Ultrasound images, magnetic resonance imaging slices, computed tomography scans, X-ray images and other medical images obtained through medical imaging technologies have different problems of speckle noise and low contrast, which seriously affect the accuracy of diagnosis. However, the currently proposed deep learning models can only address the single issue of the single-type slices and seriously ignore the temporal features, resulting in a lack of generalizability. Therefore, to solve the aforementioned limitations, a spatiotemporal cross-recognition network (STCR-Net) with robust generalization is proposed in this paper and validated on different types of medical images. The dualmodal spatiotemporal encoder is first constructed to achieve the indepth extraction of real long short-term temporal sequence features and global spatial features. Subsequently, a spatiotemporal crossskip mechanism is developed to realize the sharing of temporal and spatial feature weights. Finally, a spatiotemporal hybrid attention mechanism is designed to extract spatiotemporal fusion features that are closer to the lesion areas. Compared to other advanced classification frameworks, extensive experimental results have proved that the STCR-Net assists in solving different problems of multi-type medical images and has promising prospects for clinical application because of the state-of-the-art classification performance and robust generalization ability.