SLM-based Hybrid Retrieval for Resource Constrained Retrieval-Augmented Generation on Open Super-Large Crawled Data

Large Language Models (LLMs) have significantly advanced the accessibility of natural language processing across diverse applications. However, susceptibility to hallucinations and reliance on outdated data limit their reliability. Retrieval-Augmented Generation (RAG) systems address this by providing access to external, up-to-date information during inference. Despite this improvement, existing dense, sparse, and hybrid retrieval strategies often struggle to balance semantic relevance and efficiency. In this study, we implement a hybrid retrieval approach that utilizes a small language model (SLM)-based reranking. We compare its performance against traditional hybrid retrieval methods combining FAISS HNSW with BM25 and TF-IDF reranking on a 500,000-document subset from OSCAR, evaluated under realistic compute limits using a single-GPU Colab environment (L4 GPU, 22.5 GB VRAM) to simulate resource-constrained RAG settings. We evaluate retrieval quality (precision, Recall) and efficiency (seconds-per-query, query-per-seconds). Results show that hybrid BM25 (0.05 SPQ, 21.96 QPS) and TF-IDF (0.01 SPQ, 71.17 QPS) methods are more efficient, but the SLM-based hybrid achieves superior retrieval accuracy. It substantially outperforms both traditional approaches in top-5 relevance, achieving a Precision@5 of 0.028 and a Recall of 0.102.