Statistical Learning Using Hierarchical Modeling of Probability Tensors

Estimating the joint distribution of data sampled from an unknown distribution is the holy grail for modeling the structure of a dataset and deriving any desired optimal estimator. Leveraging the mere definition of conditional probability, we address the complexity of accurately estimating high-dimensional joint distributions without any assumptions on the underlying structural model by proposing a novel hierarchical learning algorithm for probability mass function (PMF) estimation through parallel local views of a probability tensor. This way the overall problem of estimating a joint distribution is divided into multiple subproblems, all of which are conquered independently by applying regional low-rank non-negative tensor models using the Canonical Polyadic Decomposition (CPD). Using conditioning, such parallelization is possible without losing sight of the full model - which can be reconstructed from the local models and the conditional probabilities. We illustrate the effectiveness and potential of our approach through judicious experiments on real datasets.