Generalized orthogonalization: a unified framework for Gram–Schmidt orthogonalization, SVD and PCA

This paper contributes to the understanding of orthogonalization approaches widely used in system identification, signal processing, machine learning, and automation. Generalized orthogonalization is proposed that provides a unified, alternative formulation to Gram–Schmidt orthogonalization, Singular Value Decomposition, and Principal Component Analysis over finite-dimensional Euclidean spaces. The proposed approach puts SVD and PCA into the perspective of operations research, providing (a) additional insights into their distinctive features and (b) foundations for the extensions to inner product and probability spaces.