Bayesian retrieval in associative memories with storage errors

It is well known that for finite-sized networks, one-step retrieval in the autoassociative Willshaw net is a suboptimal way to extract the information stored in the synapses. Iterative retrieval strategies are much better, but have hitherto only had heuristic justification. We show how they emerge naturally from considerations of probabilistic inference under conditions of noisy and partial input and a corrupted weight matrix. We start from the conditional probability distribution over possible patterns for retrieval. We develop two approximate, but tractable, iterative retrieval methods. One performs maximum likelihood inference to find the single most likely pattern, using the conditional probability as a Lyapunov function for retrieval. The second method makes a mean field assumption to optimize a tractable estimate of the full conditional probability distribution. In the absence of storage errors, both models become very similar to the Willshaw model.