A conditional probability interpretation of Kanerva's sparse distributed memory

It is shown that P. Kanerva's sparse distributed memory (SDM) (Sparse Distributed Memories, Bradford Books/MIT Press, 1988) is a Monte Carlo approximation to a multidimensional conditional probability integral. The SDM will produce acceptable responses from a training set when this approximation is valid, that is, when the training set contains sufficient data to provide good estimates of the underlying joint probabilities and there are enough Monte Carlo samples to obtain an accurate estimate of the integral. This analysis makes clear that in order to construct, through examples, an analog device that will transform a set of inputs into a set of outputs using the SDM, care must be taken to match the training set data to the resources. In particular, ambiguities in amplitude space can arise in a manner similar to the way aliasing can corrupt samples of analog signals in the space and time domains if the sampling density is too low. This observation could very well apply to other learning systems.<
>