Optimization transfer for computational learning: a hierarchy from f-ICA and alpha-EM to their offsprings

Likelihood optimization methods for learning algorithms are generalized and faster algorithms are provided. The idea is to transfer the optimization to a general class of convex divergences between two probability density functions. The first part explains why such optimization transfer is significant. The second part contains derivation of the generalized independent component analysis (ICA). Experiments on brain fMRI maps are reported. The third part discusses this optimization transfer in the generalized expectation-maximization (EM) algorithm. Hierarchical descendants to this algorithm, such as vector quantization and self-organization, are also explained.