Joint Probability Estimation U sing Tensor Decomposition and Dictionaries

In this work, we study non-parametric estimation of joint probabilities of a given set of discrete and continuous ran-dom variables from their (empirically estimated) 2D marginals, under the assumption that the distribution could be approxi-mated by a mixture of product densities/mass functions. Estimation of joint probability density function using semi-parametric techniques such as Gaussian Mixture Models (GMMs) is widely studied. However, they yield poor results when the underlying densities are mixtures of various other families such as Laplacian, generalized Gaussian, uniform, etc. Further, GMMs are not the best choice to estimate distributions which are hybrid in nature, i.e., when it contains both discrete and continuous components. We present a novel approach for estimating the distribution using ideas from dictionary representations in signal processing coupled with low rank tensor decomposition. We create a dictionary of various families of distributions by data inspection, and use it to approximate each decomposed factor of the product in the mixture. Our approach can naturally handle hybrid N-dimensional distributions. We test our approach on a variety of synthetic and real datasets to demonstrate its effectiveness in terms of better classification and lower error rates, when compared to state of the art estimators.