A Well-Designed Regularization Scheme for Latent Factorization of High-Dimensional and Incomplete Water-Quality Tensors from Sensor Networks

Water quality (WQ) monitoring by a network of sensors has been widely deployed in large water bodies over the last decade. Due to various reasons such as sensor failure, there are often a large number of missing values in the data collected, which greatly deteriorate the performance of WQ assessment, prediction and early warning. Numerous missing value imputation approaches are proposed to mitigate the impact of incomplete original data on follow-up applications, among which latent factorization of tensors (LFT) is proved efficient. An LFT-based model guarantees complete and correct statistics by expressing the original data as a high-dimensional and incomplete (HDI) tensor, and completing the unknown entries of an HDI tensor. However, the existing LFT-based models fail to fully account for the imbalanced distribution of known data in an HDI WQ tensor, resulting in executing LFT on an HDI tensor being ill-posed. To solve this thorny problem, this paper proposes an LFT with well-designed regularization (LWR) model. It adopts an advanced regularization scheme to enhance the model’s robustness and generality. It also applies the density-oriented strategy to establish the regularization terms to clearly express the known data distribution of an HDI tensor to achieve fine-grained modeling of regularization effects. Furthermore, it designs a novel vectorized alternating least squares (VALS) algorithm to alleviate the storage and compute costs of the training process. Experimental results on two high-frequency WQ datasets from real applications demonstrate that compared with state-of-the-art models, a LWR model achieves prominent gain in imputation accuracy for unobserved WQ data and realizes highly competitive computational efficiency.