Symbolic manipulation based on deep neural networks and its application to axiom discovery

Symbolic reasoning is difficult for neural networks. Especially, reasoning with variables can be a challenging task for them. In this paper, a symbolic reasoning method based on deep neural networks is proposed, and this method is applied to axiom discovery. This method makes use of the concept of “symbolic manipulation”. Specifically, it relies on the learning ability of the deep neural networks and the reasoning ability of a logical system: The logical system generates training examples, which indicate how to manipulate symbols, from given data, and then the deep neural networks try to learn these examples, score them and abstract possible axioms from them. In particular, this method enables the deep neural networks to realise simple reasoning with variables in predicate logic. In experiments, we demonstrate that the deep neural networks are able to learn to copy and generate symbols from a certain form of rules produced by the logical system. Moreover, we find that the more hidden layers usually mean the stronger learning ability of symbolic manipulation: An increasing number of hidden layers usually bring about a higher rule acceptance rate. Also, we find that the more hidden layers can bring about better results on axiom discovery tasks, and we show that the deep neural networks can discover some useful axioms in mathematics.