Blind signal separation revisited

Complex control and decision systems are very often confronted with an extensive amount of information about their environment from various sensors such as video cameras, etc. Hence, extraction of non-redundant signals from the available sensor information has become an important task in many control and decision problems. If the non-redundant signal extraction is based solely on a statistical method without a prior knowledge of the resulting signals, it is usually addressed as a blind signal separation. This paper provides a detailed and rigorous analysis of the two commonly used methods for blind signal separation: linear independent component analysis (ICA) posed as a direct minimization of a suitably chosen redundancy measure and information maximization (InfoMax) of a continuous stochastic signal transmitted through an appropriate nonlinear network. The paper shows analytically that ICA based on the Kullback-Leibler information as a redundancy measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance and nonlinear ICA. The practical issues of applying ICA and InfoMax are also discussed and illustrated on the problem of extracting statistically independent factors from a linear, pixel by pixel mixture of images.