Deep Vision Transformers for Prognostic Modeling in COVID-19 Patients using Large Multi-Institutional Chest CT Dataset

The importance of prognosis is the assessment of the disease progression, providing more effective management, decreasing mortality, and lowering the time of hospital stay. In convolutional neural network (CNN)-based algorithms, explicit long-range and global relation modeling is a major challenge because of the locality of convolution operations. These challenges result in weak performance because of large inter/intra-patient variabilities, specifically in COVID-19 patients. Transformers successfully used in natural language processing (NLP) tasks could potentially address the limitation of CNN-based algorithms. In the current study, we evaluated deep transformers-based algorithms’ performances in the prognostication of COVID-19 patients. Patient data from 19 centers were enrolled in this study. After inclusion and exclusion criteria, 2339 patients remained (1278 alive and 1061 deceased). We implemented a pure Transformer that consists Swin Transformer block. Images split into non-overlap patches (4 by 4) are used as tokens by the patch partitioner, followed by a linear embedding layer. These tokens pass through the patch merging layer and the Swin transformer block (feature representation learning). In addition, we implemented CNN-based algorithms, including ResNet-18, ResNet-50, ResNet-101, and DensNet, for comparison. Data were split into train/validation (70%) and test sets (30%), and all evaluations were performed on test sets. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were reported for the test sets (unseen during training). Regarding AUCs, 0.66, 0.75, 0.72, 0.77, and 0.81 were achieved by ResNet-18, Resnet-50, Resnet-101, DensNet, and Transformers.Considering all parameters Transformer significantly (p-value <0.05) outperformed all CNN-based models with Accuracy, Sensitivity, Specificity, and AUC of 0.75, 0.78, 0.71, and 0.81, respectively.