Graph-MoE: A Graph-Based LLM Framework for Scalable and Efficient Code Completion in Integrated Development Environment

Modern AI-powered integrated development environments (IDEs) face significant scalability challenges with large codebases due to the fixed context windows and high computational costs associated with large language models (LLMs). This often results in incomplete code suggestions and prohibitive operational expenses. To overcome these limitations, we introduce Graph-MoE, a novel code completion framework leveraging graph-based code representations (abstract syntax trees, data/-control flow graphs) combined with a dynamic, tiered Mixture-of-Experts (MoE) strategy using open-source models. Graph-MoE routes queries to specialized models based on context complexity: StarCoder2-3B for token-level tasks, DeepSeek-Coder-7B for function logic, and Qwen2.5-Coder-32B for architectural analysis. Our evaluations on large Python and Java codebases (up to 12M tokens) demonstrate significant improvements: Graph-MoE achieves a 72% BLEU-4 score, surpassing GitHub Copilot’s 60% baseline by 12%, maintains an average latency of 142ms through optimized routing and hybrid caching (61% hit rate), and achieves a 65% reduction in GPU costs compared to monolithic LLM approaches. These results highlight Graph-MoE’s ability to deliver highly accurate, efficient, and scalable code completion, making it a practical solution for enhancing developer productivity in modern IDEs.