Anomaly Detection in Medical Images Using Encoder-Attention-2Decoders Reconstruction

Anomaly detection (AD) in medical applications is a promising field, offering a cost-effective alternative to labor-intensive abnormal data collection and labeling. However, the success of feature reconstruction-based methods in AD is often hindered by two critical factors: the domain gap of pre-trained encoders and the exploration of decoder potential. The EA2D method we propose overcomes these challenges, paving the way for more effective AD in medical imaging. In this paper, we present encoder-attention-2decoder (EA2D), a novel method tailored for medical AD. Firstly, EA2D is optimized through two tasks: a primary feature reconstruction task between the encoder and decoder, which detects anomalies based on reconstruction errors, and an auxiliary transformation-consistency contrastive learning task that explicitly optimizes the encoder to reduce the domain gap between natural images and medical images. Furthermore, EA2D intensely exploits the decoder’s capabilities to improve AD performance. We introduce a self-attention skip connection to augment the reconstruction quality of normal cases, thereby magnifying the distinction between normal and abnormal samples. Additionally, we propose using dual decoders to reconstruct dual views of an image, leveraging diverse perspectives while mitigating the over-reconstruction issue of anomalies in AD. Extensive experiments across four medical image modalities demonstrates the superiority of our EA2D in various medical scenarios. Our method’s code will be released at https://github.com/TumCCC/E2AD.