ICA and IVA: Theory, connections, and applications to medical imaging

Independent component analysis (ICA) uses a simple generative mo- del and decomposes a given set of observations based on the assumption of statistical independence of the underlying components/ latent variables. To achieve this task, ICA makes use of the diversity in the data, typically in terms of statistical properties of the signal. Most of the ICA algorithms introduced to date have considered one of the two types of diversity: non-Gaussianity—i.e., higher-order-statistics—or, sample dependence. A recent generalization of ICA, independent vector analysis (IVA), generalizes ICA to multiple data sets and adds the use of one more diversity, dependence across multiple data sets for achieving an independent decomposition, jointly across multiple data sets. In this paper, we use mutual information rate as the unifying framework such that all these statistical properties—types of diversity—can be jointly taken into account for achieving the independent decomposition and discuss the properties of ICA and IVA under this broad umbrella.