Supercharging Document Composition with Generative AI: A Secure, Custom Retrieval-Augmented Generation Approach

Recent advancements in Retrieval Augmented Generation (RAG) provide opportunities for more efficient composition of long-form, domain-specific documents. However, few user-friendly applications leverage RAG for this specific use case. Additionally, existing RAG frameworks reliant on cloud-based, closed-source solutions pose challenges such as transparency and data privacy concerns. To address this gap, we introduce a full-stack application for automated document drafting, offering either proprietary (GPT-3.5-Turbo) or open-source (Falcon-RW-1B-Chat) Large Language Models (LLMs) for document writing and a secure, self-hosted search index (Elasticsearch) for knowledge retrieval. Users interact via a simple UI in which they submit a request with a desired topic and document type and receive a well-researched document that conforms to specific formatting and structural requirements. Given a user request, our document search pipeline employs open-source encoder models from SentenceTransformers for vector search and semantic re-ranking, then passes relevant knowledge sources to the LLM as context for document composition. To orchestrate and modularize our application backend, we use Haystack, an advanced machine learning pipeline builder, and FastAPI, which enables us to deploy components of our pipeline as independent services. Our experiments demonstrate that, using highly customized, self-hosted components, we can achieve document generation quality comparable to that of fully online RAG pipelines. This enables us to provision different application versions to accommodate different cost, scale, and data security preferences.