Joint Extended Factor Analysis

Classical Factor Analysis decomposes a covariance matrix into a low-rank and a diagonal part. In this paper we will extend this idea in two ways. We will study the case where we have more than one covariance matrix, and these matrices share the diagonal component. We will also extend Factor Analysis by replacing the diagonal part with a more general data structure. We will solve this problem via non-linear optimization, where we will exploit the Kronecker structure and arrive at a Newton-Krylov based algorithm. We will also provide an algorithm to find the Maximum-Likelihood solution taking advantage of Krylov based solvers.