Applying Self-Consistency Prompting for Knowledge Tracing with Large Language Models

Large Language Models (LLMs) offer significant advantages in Natural Language Processing, leading to recent studies on their application in Knowledge Tracing (KT). By lever-aging LLMs' pre-trained knowledge and incorporating student interaction data into prompts, the cold-start problem can be alleviated, enabling effective student modeling with limited prior information. In this study, we investigate how self-consistency prompting can enhance KT in real-world teaching scenarios. To this end, we utilize four types of datasets, each containing learning processes for solving exercises in a fixed order within specific mathematical domains. Given the limited size of these datasets, a training-free approach using LLMs is particularly beneficial. Our first experiment applies self-consistency prompting and average aggregation to KT, demonstrating improved AUC. The second experiment explores the impact of additional textual information-such as exercise statements, solution processes, correct answers, and image descriptions-on performance. Our findings highlight the potential of LLM-based KT models in adaptive learning environments and their ability to generalize with limited training data.