Sparse mixture conditional density estimation by superficial regularization

In this paper, the estimation of conditional densities of continuous random variables from noisy samples is considered. The conditional densities are modeled as heteroscedastic Gaussian mixture densities allowing for closed-form solution of Bayesian inference with full densities. The key idea is a regularization based on the curvature of the conditional density function's surface. The main contributions are the introduction of a superficial regularizer, the consideration of model uncertainty relative to the local data distribution by means of adaptive covariances, and an efficient distance-based estimation algorithm leading to an improved generalization quality of the estimates. The proposed algorithm is an iterative two-step optimization scheme for hyperparameters and the components' parameters. The obtained solutions are sparse, smooth, and generalize well as experiments, e.g., in nonlinear filtering, show.