User-Centric Evaluation of Novelty and Explanation Aspects of Recommender Systems in an Indonesia E-commerce Platform Based on Perceived Usefulness

In e-commerce, recommender systems have been widely used to help their customers with which item to buy. Keeping recommendations accurate to users has always been the main topic in recommender systems studies. However, the recommendations that are very accurate at times are not the most useful recommendations to users. Among many aspects, novelty is becoming a more common component of evaluation practice, and explanation can play a useful role. On top of that, a few researchers suggested studying recommender systems from a user-centric perspective. This research performed a user-centric evaluation to examine the influence of novelty and explanation on perceived usefulness using a within-subject design approach. During within-subject design, participants were shown two recommendation list designs from each aspect. Put simply, there are four designs in this study, and then later, participants evaluate the design. The results of the user-centric evaluation were analyzed using a wilcoxon test. The outcome from the test showed that there is no significant difference between the two novelty scenarios on perceived usefulness. On the other hand, there are significant differences between the two explanation scenarios on perceived usefulness. The conclusion of this paper is that both novelty scenarios are useful from the user perspective. In sharp contrast, one of two explanation scenarios has better influence on perceived usefulness and is more helpful based on the user’s perspective.