Turing equivalence of neural networks with second order connection weights

In principle, a potentially infinitely large neural network (either in number of neurons or in the precision of a single neural activity) could possess an equivalent computational power to a Turing machine. The authors show such an equivalence of Turing machines to several explicitly constructed neural networks. It is proven that for any given Turing machine there exists a recurrent neural network with local, second-order, and uniformly connected weights (i.e., the weights connecting the second-order product of local 'input neurons' with their corresponding 'output neurons') which can simulate it. The numerical implementation and learning of such a neural Turing machine are also discussed.<>