Certified Local Transferability for Evaluating Adversarial Attacks

Deep neural networks are known to be vulnerable to adversarial examples. Though adversarial attacks show effectiveness in misleading models, most attack methods are designed to poison a specific image. To investigate the actual effect on the feature space, we introduce the concept of the certified local transferable region. This is a connected area of inputs where we can mathematically guarantee that a single adversarial perturbation will successfully fool the model. The size of this region is a metric to evaluate the local transferability of perturbations. We present a novel method, RAOS, to estimate the maximum size of this region. Our approach efficiently searches for the largest possible vulnerable area around an original input by iteratively refining its boundaries. Each step is guided with a minimal distance attack algorithm and proven with state-of-the-art verifiers. We conduct empirical experiments to evaluate various attacks on different model structures and adversarial training scenarios.We show the advantage of our proposed metric over existing ones and demonstrate its utility in exploring the robustness of neural networks.