Improving The Local Stability of Deep Model With Margin Losses

One of the serious problems encountered by deep models is their high sensitivity to small perturbations on the input samples. It is mainly manifested in two aspects, one is the problem of adversarial samples and the other is that the gradient interpretation results of similar samples are quite different, which seriously affects the reliability and dependence of the model. In this paper, we first obtain a new loss by deriving an upper bound of the expectation of the CE loss on the local Gaussian sampled data, which is called Margin Losses. Further, under the inspiration of this margin loss and in order to improve the computational efficiency, we designed another Margin Losses based on the loss gradient. The idea of these two Margin Losses is to force samples away from the decision-making boundary to ensure local stability. Empirical results on MNIST, CIFAR-IO, and CIFAR-IOO demonstrate that our losses can indeed improve the model's resistance to adversarial noise, and at the same time can promote the local consistency of the model gradient.