Quadratic programming for learning sparse codes

Olshausen and Field (1996) used a neural network, capable of discovering sparsely distributed representations by using the principle of redundancy reduction, for the efficient coding of natural images. They showed how the resulting response functions of the units relate to the properties of simple cells in the mammalian primary visual cortex. In order to model the function of later stages of visual processing in mammals, the activation patterns of this network could be used as an input to another one of similar architecture. It would therefore be advantageous if these patterns could be calculated fast. Moreover, not only speed but also accuracy would be an important issue if the network was to be used in practical applications, such as image compression. We have derived an algorithm that achieves both goals with far less computational effort than gradient descent based minimizers.