Using learning and searching approach to explain neural network with distributed representations

The artificial neural networks have been proven useful in a variety of real-world scenarios. However, concepts learned by neural networks are very difficult to understand. Rule extraction can offer a promising perspective to provide a trained connectionist architecture with explanation power and validate its output decisions. In this paper, we present a novel approach, learning-based/search-based algorithm (LBSB), composed of two phases to extract rules from a three-layer backpropagation neural network with distributed representation. This approach combines learning and searching techniques together. Some experiments have demonstrated that the fidelity of the rules extracted from a neural network with distributed representations in our method is higher than that in conventional search-based methods, such as KT algorithms, and our method generates rules of better performance than the decision tree approach in noisy conditions.