Revisiting Gradient Regularization: Inject Robust Saliency-Aware Weight Bias for Adversarial Defense

Despite regularizing the Jacobians of neural networks to enhance model robustness has directly theoretical correlation with model prediction stability, a large defense performance gap exists when compared to the empirically perturbation-based adversarial training e.g. PGD-based, which enjoys nice discriminative saliency maps as well. To mitigate this issue, in this paper we first analyze the dilemma that the gradient map of its resulting model has no content hierarchy to mark out salient profile of input, as a negative signal of the obstructive for effective adversarial defense. Based on this, we argue that incorporating robust gradient-based saliency properties into regularized training may be helpful to reduce the performance gap. Specifically, we propose a simple method called Saliency-aware Gradient Regularization (SAGR), where a biased weight distribution strategy is introduced on positive gradient to structure and increase the impact of class-gradient components inside the Jacobian of model. The strategy maintains the dominant role of saliency-critical true-class gradient in learning process and differentiates diverse importance of gradient sensitivities that would localize input salient areas. Herein we interpret the sharpness of true-class sensitivity as robust recognition of more learning-relevant features e.g., regions containing dominant object in image for classification. Instead, false-class parts are considered as recognition-irrelevant nuisance factors e.g. the backgrounds, which are thus depressed with more strength. Experimental results demonstrate the efficacy of the proposed method and validate that distinguishment of sensitivities could further yield more robustness gain and sharper gradient saliency map.