Maximum partial likelihood methods for nonlinear signal processing

Partial likelihood (PL) establishes a general framework to develop and study statistical properties of nonlinear techniques in signal processing. Adali et al. (1997) presented the theorem by which the fundamental information-theoretic relationship for learning on the PL cost, the equivalence of likelihood maximization and relative entropy minimization, is established. In this paper, we reformulate the theorem to incorporate both the continuous and discrete probability modeling. We further show that, in both cases, the two conditions of the theorem are satisfied for the basic class of probability models, the exponential family, which includes many important network structures that can be effectively used as probability models. Hence we provide the prospect of using the PL cost in a wide class of applications with different models. We also propose several algorithms for learning/estimating the optimal model parameters by PL maximization. We give examples to illustrate the application of our general formulation and the corresponding learning algorithms.