Medical-Expert Eye Movement Augmented Vision Transformers for Glaucoma Diagnosis

Intelligence (AI) in expediting glaucoma detection and enabling consensus. The Vision Transformer (ViT) model is a promising solution for this problem as it uses the self-attention mechanism to improve performance and interpretability. Furthermore, eye-tracking data provides valuable information about a clinician's decision-making process during the diagnosis of glaucoma using Optical Coherence Tomography (OCT) reports. In this study, two approaches were originated for incorporating eye-tracking data into the ViT's training process, using solely eye movement fixation order and attention-alignment loss. Fixation-order-informed (FOI) ViT models were found to perform better than the original ViT model, with fewer parameters and faster training. The use of attention-alignment in the ViT loss function resulted in improved performance when the effect of clinician-generated spatial attention was increased. The attention maps generated by these modified ViTs enabled interpretability and made the reasons for missed predictions more transparent especially for our FOI ViT model. Overall, these findings demonstrate the potential of using expert eye-tracking data to improve the performance of ViT models in glaucoma diagnosis.