Exploring the Importance of Pretrained Feature Extractors for Unsupervised Anomaly Detection and Localization

Modeling the distribution of descriptors obtained by pretrained feature extractors is a popular approach for unsupervised visual anomaly detection. While recent work primarily focuses on the development of new methods that build on such extractors, the importance of the selected feature space itself has not been sufficiently studied. We therefore conduct a systematic analysis of current anomaly detection methods with respect to different feature extractors, their intermediate layers, and pretraining protocols. We show that the investigated methods are highly sensitive to the particular choice of feature space. We further demonstrate that using an optimal feature selection strategy can significantly improve the anomaly detection performance, up to a point where selecting a single feature layer outperforms computationally expensive ensembling approaches.