Quantized Adversarial Training: An Iterative Quantized Local Search Approach

Studies find that deep learning models are vulnerable to deliberate adversarial manipulations by attackers. Adversarial training is an effective approach to address this problem. Previous works quantize the input sample space to find appropriate perturbations on the benign samples so as to generate adversarial samples for adversarial training. However, since only the input sample space is quantized with the perturbation space being still continuous, finding the optimal perturbation noise is still a non-convex and computationally expensive problem. Moreover, in this case, the found perturbation noise that will be used to generate an adversarial sample may be strong in the continuous search space, but may become weak after quantization in the input sample space. In this paper, we first develop an Iterative Quantized Local Search (IQLS) algorithm that finds strong perturbation noises by quantizing both the input space and perturbation space. Then, we theoretically analyze and prove the upper bound on the number of iterations needed for the IQLS algorithm, based on which we devise an efficient and effective Quantized Adversarial Training (QAT) scheme. Experiment results on six public datasets show that our proposed scheme outperforms state-of-the-art methods to defend against different adversarial attacks. Particularly, QAT improves the system performance by 14%, 11%, 16% on average on CIFAR-10, SVHN, and CIFAR-100 datasets respectively compared with the existing defense schemes, and reduces the computing time by about 60%.