Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities

It is well known that Boltzmann machines are nonregular statistical models. The set of their parameters for a small size model is an analytic set with singularities in the space of a large size one. The mathematical foundation of their learning is not yet constructed because of these singularities, though they are applied to information engineering. Recently we established a method to calculate the Bayes generalization errors using an algebraic geometric method even if the models are nonregular. This paper clarifies that the upper bounds of generalization errors in Boltzmann machines are smaller than those in regular statistical models.