Decentralized Rank-Adaptive Matrix Factorization — Part I: Algorithm Development

Factorizing a low-rank matrix into two matrix factors with low dimensions from its noisy observations is a classical but challenging problem arising from real-world applications. This paper develops decentralized matrix factorization algorithms, i.e., factorizing a matrix whose columns are stored distributively over a network without a central agent. The performance of existing algorithms relies heavily on the accuracy of the matrix rank estimate. However, acquiring an accurate estimate is difficult in the distributed setting. In this paper and its Part II, we address this problem by introducing a novel regularization into the objective function to induce the solution with correct rank. Based on this, we propose a rank-adaptive decentralized MF algorithm. In Part I, we delineate the algorithm development from the centralized with known rank, decentralized with known rank, to the rank-adaptive decentralized settings. For the centralized algorithm, we present the first globally linear convergence analysis for the alternating gradient descent method. In the Part II, we analyze conditions for which the proposed rank-adaptive decentralized MF algorithm converges to the global solution with the correct rank. Numerical experiments based on both synthetic and real-world datasets are presented in this paper to demonstrate the effectiveness of the proposed algorithms and corroborate the theoretical claims.