WebUltron: An Ultimate Retriever on Webpages Under the Model-Centric Paradigm

Document retrieval has been extensively studied within the
index-retrieve
framework for decades, which has withstood the test of time. However, this approach inherently segregates the indexing and retrieval processes, preventing a cohesive, end-to-end optimization. To bridge this divide, we introduce WebUltron, a revolutionary model-centric indexer for document retrieval. This system embeds the entirety of document knowledge within the model, striving for seamless end-to-end retrieval. Two primary challenges with this indexer are the representation of document identifiers (docids) and the model's training. Current methods grapple with docids that lack semantic depth and the constraints of limited supervised data, making scaling up to larger datasets challenging. Addressing this, we've engineered two novel docid types imbued with richer semantics that also streamline model inference. Further enhancing WebUltron's capabilities, we've developed a three-stage training regimen, leveraging deeper corpus insights and fortifying query-docid relationships. Experiments on two public datasets demonstrate the superiority of WebUltron over advanced baselines for document retrieval.