Evolving neural network models

Neural networks in nature are not designed but evolved, and they should learn their structure through the interaction with their environment. The paper introduces the notion of an adaptive neural network model with reflection. We show how reflection can implement adaptive processes, and how adaptive mechanisms are actualized using the concept of reflection. Learning mechanisms must be understood in terms of their specific adaptive functions. We introduce an adaptive function which makes the network able to adjust its internal structure by itself to by modifying its adaptive function and associated learning parameters. We then provide the model of emergent neural networks. We show that the emergent neural network model is especially suitable for constructing large scale and heterogeneous neural networks with the composite and recursive architectures, where each component unit is modeled to be another neural network. Using the emergent neural network model, we introduces the concepts of composition and recursion for integrating heterogeneous neural network modules which are trained individually.