The Impact of LLM Hallucinations on Motor Skill Learning: A Case Study in Badminton

The rise of Generative Artificial Intelligence, including Large Language Models (LLMs), has enabled users to engage in self-guided learning of sports skills through conversation-based interactions. However, studies have identified a phenomenon known as “hallucination” in which LLMs generate feedback that is inaccurate or non-existent. While this phenomenon has been observed in various domains, including medicine, academia, and news, its existence and implications in the context of physical exercises, particularly motor skill learning, remain unexplored. This study investigates the presence of LLM hallucinations in badminton skill learning and examines their potential impact on learning outcomes. This study aims to investigate whether LLMs hallucinations exist in the motor skill learning of physical exercises and what impact they may have. Eighty university freshmen with no prior badminton experience participated in a 16-week experiment, with 40 students assigned to the Experimental Group (EG) utilizing LLM-based applications (ChatGPT or New Bing) for self-guided learning, and 40 students in the Control Group (CG) learning under the supervision of 12 university sports teachers and 8 experts that specialized in badminton. Evaluation criteria for badminton skills were established, and assessments were conducted at baseline and 16 weeks using independent sample t-tests and paired-sample t-tests. One-way analysis of variance (One-Way ANCOVA) was employed to compare learning outcomes between the two groups. Interviews were conducted to gain insights into the causes of any observed differences in learning efficiency. Both CG and EG groups demonstrated motor skill improvement (clear: p <0.001; smash: p <0.001; footwork: p <0.001). CG exhibited significantly higher scores in long-distance shots and smashes in the post-test. No significant difference was observed in footwork scores between the two groups. High accordance in specific skill points among students in both groups indicated the common usage of prompts. Interviews with EG students revealed hallucinations in the text generated by LLMs, particularly in the context of “forearm internal rotation swing.” LLMs exhibit hallucinations in the context of intricate motor skill learning, such as badminton, where limited corpus data is available. These hallucinations can mislead users and impact learning outcomes. Future research should explore strategies to mitigate LLM hallucinations in physical exercise learning applications.