A Novel Cross Language Neural Retrieval Model

Cross-language information retrieval is one of the important tasks in the field of information retrieval. Existing cross-language neural retrieval methods usually use single-task learning, and the single feature capture model limits the performance of neural retrieval models. To this end, a cross-language retrieval method based on multi-task learning is proposed, using a text classification task as a secondary task, capturing feature information of both tasks simultaneously using a shared text feature extraction layer so that it learns the feature patterns of different tasks, and then inputting the feature vectors into the neural retrieval model and the text classification model to complete the two tasks, respectively. In addition, the external corpus introduced by the text classification task also plays a role in data augmentation to a certain extent, further increasing the level of feature information. Experiments conducted on four language pairs from the CLEF 2000–2003 dataset show that the present method significantly improves the text feature extraction and thus enhances the neural retrieval model performance, increasing the MAP values of the neural retrieval model by 0.014-0.203 and increasing the model convergence speed by 22.82% on average.