Serverless RAG-Stream: A Cloud Pipeline for Efficient Real-Time Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating external knowledge through retrieval mechanisms, thereby improving the contextual relevance and accuracy of generated responses. However, conventional RAG systems often encounter challenges related to high latency, limited scalability, and the complexity of managing infrastructure particularly in real-time streaming environments. To overcome these barriers, this work proposes a novel serverless RAG-Stream architecture, designed to deliver low-latency, highly scalable, and cost-effective data processing. The architecture employs AWS Kinesis for real-time data ingestion, Dense Passage Retrieval (DPR) for semantic search, and cloud-based LLMs for dynamic response generation. Workflow orchestration is achieved using serverless functions and asynchronous communication, ensuring optimal resource utilization and simplified management. Experimental evaluations, performed using the WikiText-103 dataset alongside four subsets of the Pile benchmark such as Enron Emails, HackerNews, NIH ExPorter, and YouTube Subtitles, reveal significant performance gains. The serverless RAGStream system achieved an average latency of 150 ms, sustained a throughput of 500 requests per second, and produced highquality outputs, as validated by BLEU, ROUGE, METEOR, and BERTScore metrics. Compared to baseline models, the proposed architecture demonstrates superior scalability, reduced operational complexity, and maintained generation quality, positioning it as a robust solution for real-time, AIpowered applications.