Efficient CLIP Adaptation for Video Anomaly Detection via Spatial and MLP Adapters

In this paper, we propose a novel integration of Contrastive Language-Image Pretraining (CLIP) with two innovative adaptation modules—the Spatial Adapter and the MLP Adapter—for weakly-supervised video anomaly detection. While CLIP offers strong cross-domain generalization, its feature representations lack fine-grained discrimination, and its high computational cost limits adaptability. To address these challenges, our lightweight adaptation framework refines CLIP’s feature space while significantly reducing the number of parameters that need to be fine-tuned, improving both adaptability and efficiency. Additionally, our adapters enable CLIP to better capture subtle anomalies while preserving its transferability. Specifically, the Spatial Adapter employs a prompt-based mechanism to dynamically filter out normal regions, directing attention to potentially anomalous areas and enhancing feature extraction. Meanwhile, the MLP Adapter leverages cosine similarity to align training and testing features, mitigating distribution shifts from video-level labels. All together, these modules enhance CLIP’s effectiveness while reducing computational overhead. We validate our approach on the widely used XD-Violence and UCF-Crime datasets, where our framework achieves performance comparable to state-of-the-art methods while maintaining computational efficiency.