Pseudo Gradient-Adjusted Particle Swarm Optimization for Accurate Adaptive Latent Factor Analysis

A latent factor analysis (LFA) model can be efficiently built via the stochastic gradient descent (SGD) algorithm to address high-dimensional and incomplete (HDI) data generated by various big data-related applications. However, its performance depends hugely on its hyperparameters, which is conventionally decided through the grid-search that yields extremely high-computational costs. Particle swarm optimization (PSO)-based hyperparameter adaptation provides a potential solution to this severe problem, while it leads to accuracy loss due to the untimely convergence of the PSO algorithm. Moreover, existing PSO extensions mostly perplex the evolution scheme to make the hyperparameter adaptation time-consuming and computationally expensive. Aiming at implementing an accurate and adaptive LFA model, a pseudo gradient-adjusted PSO (PGA-PSO) algorithm with twofold ideas is proposed: 1) modeling the position transitions in a PSO algorithm as the pseudo gradients for optimizing each particle’s position and 2) incorporating the principle of adaptive moment estimation into the pseudo gradient estimation to make it consider previous pseudo gradient information for addressing the untimely issue. With it, a PGA-PSO-incorporated LFA (PPL) model is successfully constructed. The empirical studies on six HDI datasets demonstrate that owing to the efficient hyperparameter adaptation implemented by the proposed PGA-PSO algorithm, the obtained PPL model surpasses state-of-the-art LFA models in accuracy when predicting an HDI matrix’s missing data. Hence, it satisfies the scalability and efficiency demands emerging from practical applications.