Purifier$^{+}$: Plug-and-Play Backdoor Mitigation for Pre-Trained Models via Activation Alignment

Pre-trained models are extensively embraced in deep learning, facilitating efficient fine-tuning for downstream user-specific tasks and yielding substantial computational savings. However, backdoor attacks present a significant security threat to downstream models constructed on corrupted pre-trained models, necessitating the implementation of effective countermeasures to mitigate this threat prior to deploying the models in safety-critical applications. This paper introduces Purifier and its advanced version Purifier$^{+}$, the former of which mitigates backdoors in pre-trained models by aligning anomaly activation to normal activation, and the latter builds on this by making importance rating about activation patterns, boosting important activation patterns and suppressing unimportant activation patterns. Purifier and Purifier$^{+}$ draw inspiration from the observation that anomaly activation patterns for backdoor triggers manifest across various perspectives such as channel-wise, cube-wise, and feature-wise, each exhibiting distinct levels of granularity. Crucially, the choice of alignment granularity plays a pivotal role in ensuring robustness and accuracy. In addressing this challenge, Purifier and Purifier$^{+}$ demonstrate the ability to effectively thwart various categories of backdoor triggers devoid of requiring prior information about the specific backdoor attacks. Additionally, it offers a convenient and flexible deployment feature, namely, plug-and-play capability. The comprehensive experimental results demonstrate that Purifier and Purifier$^{+}$ outperform current methodologies regarding defense efficacy and accuracy in model inference with uncontaminated samples when subjected to a series of State-of-the-Art mainstream attacks.