Serialised Pulmonary Lesion Segmentation with Spatiotemporal Transformer

In the medical treatment of pneumonia, most patients undergo multiple CT scans at different time points, generating serialised CT data. However, traditional segmentation methods based on convolutional networks are difficult to capture inter-slice dependencies, which can result in missing sequence information. Additionally, Transformer-based segmentation methods tend to overemphasize temporal information, leading to the loss of local features in a single image. To address these issues, we propose a Serialised Pulmonary Lesion Segmentation with Spatiotemporal Transformer (SPLS-STT). Specifically, we firstly capture local features of lesion regions and global features from the serialised lung CT through a self-attention mechanism, which generates a spatial feature maps. Secondly, the obtained spatial feature maps at multiple time points are arranged in chronological order, and the dependencies between the pneumonia lesions at different time points are established by the cross-window temporal attention mechanism. Finally, we combine the features in the temporal and spatial dimensions through the cross-attention mechanism to compensate for the loss of local features during feature extraction in the temporal dimension. Experimental results show that our spatiotemporal Transformer model outperforms the traditional single-dimensional processing methods in terms of accuracy metrics. Furthermore, our model achieves a Dice coefficient of 37.9% on the lung CT sequence dataset, which is an improvement of 9.5% compared to the Vision Transformer.