QuantLLM: A Hybrid Classical-Quantum LLM Transformer with Adaptive Routing Framework for Inference Latency Minimization

Token generation in Large Language Models (LLMs) are designed on multi-head attention transformers. They capture long-range sequences inherently, which might span transformer full attention pass, and results in escalation of the model cost quadratically. This cost inflation trigger higher inference latency, limiting its deployment in resource-constrained scenarios. Quantum computing (QC) can address these limitations, owing to its advantages of entanglement in high-dimensional Hilbert spaces, and allow parallel sequence generation by LLMs. However, pure quantum transformers design are challenged by high noise fidelity, coherence, and lack of sufficient qubits for operation. Motivated by this, in this paper, we design a framework, QuantLLM, a hybrid classical-quantum transformer architecture that reduces latency via a token-wise, and semantically-aware adaptive routing mechanism. In the routing design, tokens are assigned to either classical or quantum attention paths computed by a differentiable complexity score, that minimizes the LLM inference latency. Results are evaluated on the benchmark WikiText-2 dataset. The framework reduces inference latency by a maximum of 30.06% compared to quantum-enhanced attention baselines. At a balanced setting, it achieves 84.1% accuracy with 290 ms latency, versus 85.7% with 418 ms against baseline scheme. For long sequences, up to 48.7% of tokens are routed through the quantum path, with token-level fidelity scores (0.872 for complex nouns and 0.122 for punctuation), which asserts optimized semantic-aware context routing. These results indicate the potential viability of the framework design in practical model settings.