Specifying intrinsically adaptive architectures

The paper describes a method for specifying (and evolving) intrinsically adaptive neural architectures. These architectures have back-propagation style gradient descent behavior built into them at a cellular level. The significance of this is that we can now use back-propagation to train evolved feedforward networks of any structure (provided that individual nodes are differentiable). Networks evolved in this way can potentially adapt to their environment in situ. This is in contrast to more conventional techniques such as using a genetic algorithm or simulated annealing to train the network. The method can be seamlessly integrated with any method for evolving neural network architectures. The performance of the method is investigated on the simple synthetic benchmarks of parity and intertwined spiral problems.