Progressive Stochastic Binarization of Deep Networks

We propose a stochastic binarization scheme for deep networks that approximates scalar products of weights and activations using progressive sampling of stochastic shifts. This representation has bounded relative error and thereby permits high accuracies at moderate sampling costs. Further, it allows for the first time a fully dynamic and localized control of accuracy. This not only enables a choice of accuracy at run-time, but also provides a new tool for adaptively focusing computational attention. A reference implementation is provided under a free license (https://github.com/JGU-VC/progressive_stochastic_binarization).