DKBERT: An Efficient and Effective Ranking Model via Weight Sharing Decoupling and Kernel Matching

Recently, the use of BERT has remarkably improved the performance of Information Retrieval (IR) tasks. However, the BERT model needs to take each concatenation of queries and documents in the collection as input, and its computational cost makes it less practical to use in real-time ranking scenarios. Some methods try to reduce the time cost but may suffer from the partial matching problem due to the max-sum mechanism. This paper proposes a new model named DKBERT, it first uses two encoders sharing the same weights to decouple the query encoder and the document encoder and make the query and document in the same semantic space, and then uses multiple kernels to extract different matching signals from the term interaction matrix, rather than using only the best matching score. With the decoupling mechanism, the model could precompute and cache the document representation, and different ranking features could be computed in a parallel way, so the model could create a better balance between the ranking efficiency and the ranking effectiveness. Experimental results on two benchmark datasets show that our proposed model could improve the ranking performance significantly (with 3.1% promotion at MRR@10) with less time cost.