The Dilemma of Sufficient Prediction Accuracy in Educational Recommendation Services

The study discusses the integration of technological solutions based on social media data for vocational guidance in education. It focuses on educational guidance services like «Career Guidance Robot», «Wizard», and «IOT Navigator». The analysis explores reasons for unsuccessful launches of career guidance services, emphasizing the challenge of achieving sufficient prediction accuracy and its impact on user and developer expectations. In «Wizard», for example, an accuracy of 65-67% is deemed insufficient, while 85-87% accuracy is not consistently attained across all training areas, illustrating the challenges in meeting expectations. Paradoxically, very accurate predictions may render a system ineffective, as any mistake can erode user trust, even if the system has been accurate in the past. This dilemma can be referred to as the «sufficient prediction accuracy dilemma». The effectiveness of a recommender system relies on well-defined performance targets and its future success is determined by crucial decisions regarding the inclusion of data in training, validation, and test samples. This is especially significant for recommender systems in education, where challenges like high data sparsity and non-uniformity can potentially result in the system9s disruptions.