Structure Analysis of a Probabilistic Network in an Information Geometric Framework

The structure of a probabilistic network is a crucial component of the network because it gives us insights into the underlying dependency or causal relationships among the random variables. This paper analyzes the structure of a parametric probabilistic network using information geometry. We start by parameterizing the joint distribution of the probabilistic network as an exponential family in the parameter space Theta . Our first result shows that the structure of a probabilistic network corresponds to a unique Riemannian submanifold of the parameter space. Secondly, we show that an incorrectly structured network used to estimate the parameterized joint distribution p(Xoarr;thetas), has an intrinsic bias error, which is defined as the shortest divergence between p(Xoarr;thetas) and the submanifold corresponding to the probabilistic network. Furthermore, by adopting a generalized definition of mutual information derived from information geometry, one can, in theory, extend Chow's method of constructing trees based on pair-wise mutual information to an arbitrary clique size.