Diving into Text Representation Learning with Deep Hashing

With the development of deep neural network, especially the emergence of large scale pre-trained language model, methods based on deep learning have substantially advanced the state of the art across a variety of subtasks in natural language processing. Nevertheless, in order to get better performance, the increasing model parameters and exponential growth of training time become problems that can not be ignored. In this paper, inspired by the application of deep hashing in large-scale image retrieval, text representation learning based on deep hashing technology has been extensively explored. And we use three common subtasks in natural language processing to evaluate the proposed method. Experimental results show that deep hashing can greatly reduce the physical space cost by text representation under the condition of limited performance loss, which is of great significance in large-scale text representation, and is worthy of further exploration.