A Unified Multimodal Learning Analytics Framework for Course Design with LLMs

With the rapid development of artificial intelligence (AI) technology, AI-based multimodal learning analytics has attracted significant attention in the field of education. This paper proposes a framework for optimizing course design by integrating classroom multimodal data and large language models (LLMs). The framework combines classroom expression recognition, speech recognition, and LLM analysis to perform data fusion and intelligent analysis, enabling real-time course optimization. By integrating the RT-DETR and Whisper models for temporal modeling of multimodal data and combining ChatGPT with other large language models for course design improvement and evaluation, the optimal course design is selected. Experimental results show that this framework significantly enhances the accuracy of course design and the adaptability of teaching strategies, providing new insights for course design in dynamic educational environments.