Integration of Large Language Models and Computer Vision Algorithms in LMS: A Methodology for Automated Verification of Software Tasks and Multimodal Analysis of Educational Data

This paper presents a study on the design and development of a learning management system (LMS) integrated with large language models (LLMs) and image recognition/generation algorithms to automate software task validation and multimodal educational data analysis. The purpose of the development is to improve the quality of feedback for learners, deepen the understanding of errors made and expand the capabilities of the system by synthesizing formal LLM models and visual processing algorithms. The system is based on the use of domestic LLMs such as GigaChat and YandexGPT, Retrieval-Augmented Generation (RAG) technology, and modern computer vision algorithms. The system architecture is described, including GitVerse for code storage, GitVerse Actions for automated testing of solutions in Docker containers, and image processing modules for analyzing and generating visual data in the educational process. The conducted experiment demonstrates the effectiveness of the integrative approach, which allows to improve the quality of feedback and increase the performance of students. Prospects for further research include optimization of computational resources and development of techniques for integrated analysis of multimodal educational data.