Enhancing BERT-based Passage Retriever with Word Recovery Method

Passage retrieval is a fundamental task in information retrieval research and has received extensive attention on academia and industry in recent years. BERT-based passage retriever adopts a dual-encoder architecture to learn dense representations of queries and passages for semantic matching, which has become an indispensable component of passage retrieval system. However, BERT-based passage retriever tends to ignore phrases and entities mentioned in sentences, which are critical of retrieval. To address this problem, we propose a word recovery method that introduces word-granularity information into BERT-based paragraph retriever. We evaluate our model on three public datasets in different domains including E-commerce, Entertainment Video and Medical. By augmenting BERT-based passage retriever with word recovery method, the retriever achieves consistent improvements in MRR@10 and Reca11@1000, of which MRR@10 increased by 1.2% in Entertainment Video, and Reca11@1000 increased by 0.6% in Ecommerce.