Robust Neural Networks Learning via a Minimization of Stochastic Output Sensitivity

this article, we propose Sensitivity Minimization Learning (SML) to overcome the performance degradation problem caused by features corruption at the testing phase by using the stochastic sensitivity measure (STSM) as a regularizer. The STSM measures output deviations between each training sample and its noisy versions generated by a feature perturbation strategy. The feature perturbation strategy is user defined to simulate noises that our model target to defense at the testing phase. The SML is computational efficiency in training and testing phase and minimizes the generalization error on both training samples and its noise version set with feature perturbations. Models regularized by the STSM can be trained by the stochastic gradient descent algorithm efficiently and applied to very large scale applications. Experiment results on eight grayscale image, one color image, and two face databases show that the SML significantly outperforms several regularization techniques and yields much lower classification error when testing sets are contaminated with noises.