Data-Driven Driver Training via Counterfactual and Language-Based Guidance in Racing Scenarios

Novice drivers in automobile racing face significant challenges in improving their racing performance while maintaining safety. Traditional one-on-one human coaching faces practical limitations such as subjectivity, inconsistency, high cost, and limited scalability. Recent alternatives—such as optimal racing line visualizations or predefined skill metrics—often lack personalization and fail to explain why or how drivers should improve. To address these gaps, we propose a fully data-driven training framework that delivers personalized, interpretable, and actionable feedback based on corner-specific performance. Our approach first clusters sequential driving behavior to generate pseudo skill labels in an unsupervised manner. A classification model is then trained on racing-critical features to assess performance levels for each corner. To provide targeted guidance, we introduce a novel counterfactual explanation method based on negative SHapley Additive exPlanations (SHAP) values. This identifies which feature changes would most effectively elevate performance to the expert level. Feedback is then translated into natural language using a customized large language model (LLM), enabling novice drivers to intuitively understand and apply the suggestions.We validate our system through both algorithmic evaluation and human-in-the-loop experiments in a simulated racing environment. Our clustering method effectively groups driver strategies, and the classification model achieves high skill-level prediction accuracy. Counterfactual feedback based on negative SHAP values yields actionable improvements that significantly enhance both cornering performance and overall lap times. Furthermore, participants internalized the feedback and sustained performance gains even after feedback was removed, demonstrating the system’s strong potential for scalable driver training and broader educational applications.