Addressing Technical Challenges in Large Language Model-Driven Educational Software System

The integration of large language models (LLMs) into educational systems poses significant challenges across several key attributes, including integration, explainability, testability, and scalability. These challenges arise from the complexity of coordinating system components, difficulty interpreting LLM decision-making processes, and the need for reliable, consistent model outputs in varied educational scenarios. Additionally, ensuring scalability requires robust autoscaling mechanisms and suitable architecture design to handle fluctuating workloads. This paper tackles these challenges by proposing tactics to improve system integration, enhance explainability through metadata and an algorithm process, ensure response consistency via regression testing, and facilitate efficient autoscaling through an event-driven microservice architecture. The evaluation results highlight the effectiveness of these tactics, confirming both functional consistency and robust system performance under varying loads.