Prompting the Unseen: Detecting Hidden Backdoors in Black-Box Models

Ensuring the security of AI models is becoming increasingly important as AI systems are integrated into various products and services. In the future, many AI models may undergo security evaluations at testing centers to obtain certifications that validate their security and reliability. However, due to concerns about trade secrets, companies may only provide black-box access to their models during testing. This necessitates the development of black-box model-level backdoor detection methods, since evaluation is limited to the model itself, without access to its internal structure or training data.In this work, we propose BProm, a novel black-box model-level backdoor detection method based on visual prompting (VP). VP is a technique that adapts well-trained frozen models for source domain tasks to target domain tasks by mapping class subspaces between the source and target domains. We identify a critical misalignment, termed class subspace inconsistency, between clean and poisoned datasets. Using this insight, BProm detects backdoors by observing the low classification accuracy of prompted models when backdoors are present. Extensive experiments demonstrate the effectiveness of BProm in identifying backdoored models under black-box settings, paving the way for secure and reliable certification of AI models.