A Recommender Systems’ algorithm evaluation using the Lenskit library and MovieLens databases

A wide variety of algorithms can be found in the recommender system's literature for computing predictions and obtaining recommendation lists. These algorithms' performance can be evaluated by numerous metrics depending on which business goal is intended to be optimized. However, performance can vary drastically depending on the selected algorithm as well as the selected metric. In addition, algorithmic performance can be heavily influenced by the experimental setup such as: dataset, cross-validation strategy, recommender framework and the model hyper-parameters chosen. In this work we will compare nine different algorithms used for generating recommendations across seven different metrics. One vital aspect of our analysis is to provide a detailed and clear explanation of the methodology carried out as well as the experimental setup in order to ensure future reproducibility. Moreover, performance and computation times will be introduced for further comparison. Finally, optimal model hyper-parameter combinations will be presented for benchmarking purposes.