Optimization criteria and nonlinear PCA neural networks

In this paper, the relationships of various nonlinear extensions of principal component analysis (PCA) to optimization are considered. Standard PCA arises as an optimal solution to several different information representation problems. It is claimed that basically this follows from the fact that PCA solution utilizes second-order statistics only. If the optimization problems are generalized for nonquadratic criteria so that higher order statistics are taken into account, their solutions will in general be different. The solutions define, in a natural way, several nonlinear extensions of PCA and give a solid foundation to them. The respective gradient type neural algorithms are discussed.<>