BYY dependence reduction theory and blind source separation

Bayesian Ying-Yang dependence reduction (BYY-DR) system and theory is introduced, together with a generic stochastic implementing procedure and a generic model selection criterion. Their specific forms in the forward, backward, bi-directional architectures are further elaborated. The forward one provides a general information theoretic DR scheme, which is applied to blind source separation (BSS) problems by instantaneous models, with a criterion for deciding the unknown number of sources and a new parametric mixture based implementation model obtained. The backward architecture provides a general maximum likelihood (ML) mixture model for the noise contaminated mixture of unknown number sources. The bi-directional architecture combines the advantage of backward and forward ones, and includes the existing LMSER based nonlinear PCA approach and the one hidden layer deterministic Helmholtz machine as special cases.