Neural Networks-Incorporated Latent Factor Analysis for High-Dimensional and Incomplete Data

HDI matrices are commonly encountered in a variety of big data-related industrial applications, which describe complex interactions between entities. The complete interaction relationship in the high-dimensional and incomplete (HDI) matrix is essential to deal with various problems such as pattern recognition in industrial applications. Therefore, estimating the missing data in the HDI matrix is crucial. LFA models have achieved advanced results in solving such problems. However, the existing latent factor analysis (LFA) models cannot model the nonlinear structure hidden in the HDI matrix. NN can handle the nonlinearity in the HDI data, but their high estimation accuracy relies on high computation cost and storage burden. To address the aforementioned problems, this article proposes a novel NNLFA model. It contains the following primary ideas: 1) it can model the nonlinear structure of the HDI matrix efficiently through neural networks (NNs) and 2) it incorporates the NNs into the LFA model to improve estimation accuracy while maintaining high computational and storage efficiency. To validate the superiority of the NNLFA model, experiments with six state-of-the-art models are conducted on six practical industrial application datasets. The experimental results indicate that the NNLFA model enhances estimation accuracy by up to 33.3%. In addition, NNLFA model shows strong competitiveness in terms of both time and storage efficiency when compared to baseline models.