DA2F: Research on Robustness of Deep Learning Models Using Approximate Activation Function

Deep learning structures have been adopted in many application domains. However, these architectures are vulnerable to adversarial attacks which are often instantiated by adversarial examples: carefully crafted inputs by adding disturbances that are imperceptible to humans can easily mislead a learned classifier to make incorrect predictions. Since deep learning is fast achieving the maturity to enter into safety- critical and security-sensitive applications, such attacks may have catastrophic security and safety consequences. In this paper, we propose a new software-based approach to enhance the robustness of deep learning models to adversarial attacks which we call the Defensive Approximate Activation Function (DA
²
F). Specifically, we mainly consider deep learning architectures using the sigmoid function or tanh function, two complex non-linear involving exponential operation functions, as activation functions. We propose the piecewise linear approximation method where a new non-uniform segmentation scheme is presented. By replacing the exact activation function used in the deep learning architectures with the approximate activation function, which reduces computation cost theoretically and improves deep learning models' robustness to adversarial samples. The experiments validated that our approach was effective in defending against adversarial attacks. For LeNet-5 CNN architecture along with MNIST datasets, the approximate classifiers were more robust against adversarial attacks than the exact classifiers, with negligible loss in accuracy.