CAMVR: Context-Adaptive Multi-View Representation Learning for Dense Retrieval

The recently proposed MVR (Multi-View Repre-sentation) model achieves remarkable performance for open-domain dense retrieval. In MVR, matching multi-view queries is enabled by encoding the document into multiple representations. However, these representations tend to collapse into the same one when the percentage of documents answering multiple queries in the training data is low. Here, we propose a Context-Adaptive Multi-View Representation (CAMVR) learning framework that explicitly avoids this collapse problem by aligning each viewer token with different document snippets. In CAMVR, each viewer token is placed before a snippet to capture the local and global information with consideration that answers of different view queries may scatter in one document. The view of the snippet containing the answer is also used to explicitly supervise the learning process, from which the interpretability of the view representation is provided. Lastly, this work presents extensive experiments demonstrating that CAMVR outperforms the existing models to achieve state-of-the-art results.