Constrained quadratic optimization problems with applications

Optimization problems involving strictly quadratic constraints arise in many algorithmic developments in signal processing, applied mathematics, physics, and control theory. In this paper, a framework is developed based on 1) incorporating diagonal matrices into the cost function and/or the constraints, and 2) exploiting the symmetry of quadratic constraints in the Lagrangian function. Using this framework, many algorithms for true principal and minor component extraction and true principal singular component analysis are derived from optimizing a weighted inverse Rayleigh quotient and weighted Rayleigh quotient like criteria. The main features of these algorithms are that they are self-stabilizing, can compute multiple principal, minor or singular components, and they ensure orthogonality. Additionally, using a logarithmic cost function, fast convergent power-like methods for computing principal components and singular vectors are developed. Three lists for minor component analysis, principal component analysis, and principal singular component analysis are provided.