By Topic

The Spike-and-Slab RBM and Extensions to Discrete and Sparse Data Distributions

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

The purchase and pricing options are temporarily unavailable. Please try again later.
4 Author(s)
Courville, A. ; Dept. of Comput. Sci. & Oper. Res., Univ. of Montreal, Montreal, QC, Canada ; Desjardins, G. ; Bergstra, J. ; Bengio, Y.

The spike-and-slab restricted Boltzmann machine (ssRBM) is defined to have both a real-valued “slab” variable and a binary “spike” variable associated with each unit in the hidden layer. The model uses its slab variables to model the conditional covariance of the observation-thought to be important in capturing the statistical properties of natural images. In this paper, we present the canonical ssRBM framework together with some extensions. These extensions highlight the flexibility of the spike-and-slab RBM as a platform for exploring more sophisticated probabilistic models of high dimensional data in general and natural image data in particular. Here, we introduce the subspace-ssRBM focused on the task of learning invariant features. We highlight the behaviour of the ssRBM and its extensions through experiments with the MNIST digit recognition task and the CIFAR-10 object classification task.

Published in:

Pattern Analysis and Machine Intelligence, IEEE Transactions on  (Volume:36 ,  Issue: 9 )