A Novel Learning Rate Function and Its Application on the SVD++ Recommendation Algorithm

The recommendation algorithm based on Singular Value Decomposition (SVD)++ is a widely used algorithm for its good prediction performance. However, with the rapid increase of data in smart societies, the poor computational performance of the SVD++ recommendation algorithm becomes a prominent disadvantage, for it takes a longer time to optimize the objective function during constructing the prediction model. The learning rate function is a significant factor in the prediction model based on the SVD++ recommendation algorithm. It can directly affect the convergence speed of the prediction model and the performance of the model. The traditional model uses an exponential function, natural exponential function or piecewise constant as its learning rate function. In this paper, a novel adaptive learning rate (ALR) function is proposed, which combines the exponential with linear functions, and the function is applied to the SVD++ recommendation algorithm. The highlights of the paper are as follows. First, with a larger initial value, the proposed function descends quicker and tends to the end with a less step. Second, the theoretical properties of the proposed learning rate function are verified through theoretical analysis, including the theoretical proof of its convergence and the iteration speed comparison. Compared to the existing learning rate functions, the proposed ALR function works better on the convergence speed through mathematical derivation. Finally, the novel ALR function is applied to the SVD++ recommendation algorithm as recommendation model ALRSVD++. Some existing learning rate methods are used as benchmarks for illustrating the computation and prediction performances of proposed ALR function and its ALRSVD++ model. Experimental results demonstrated that the SVD++ recommendation algorithm based on the proposed ALR function improved computational efficiency of the training model ALRSVD++ significantly. Especially, to the larger size training dataset, the iterations and training time based on the proposed ALR function and ALRSVD++ model reduced in a great deal, without greatly sacrificing the recommendation performance.