Distilling Deep Neural Networks for Robust Classification with Soft Decision Trees

Recent deep neural networks have achieved impressive performance in image classification. However, these networks are sensitive to the attack of adversarial examples, leading to a sharp drop in accuracy. To address this issue, this paper proposes a learning approach to improve the robustness by distilling deep neural networks with soft decision trees. This approach learns a decision tree in a softening manner by jointly using data and the predictions of a well-trained deep neural network. In this way, the resulting soft decision tree can distil the knowledge from deep neural network when preserving the efficiency of decision tree. Experimental results show that the proposed approach has better robustness again adversarial examples than deep neural networks and decision trees.