Hebbian reinforcement learning in a modular dynamic network

We present a multi-population dynamic neural network model with binary activation and a random interaction pattern. The weights parameters have been specified in order to distinguish excitatory populations from inhibitory populations. Under specific parameters, we design functional modules composed of two populations, one of excitatory neurons, one of inhibitory neurons. Such modules are found to display a weak chaotic activity, and to react toward incoming stimulations with increasing synchronization. We also present the design of topologically structured neural maps. We then combine such modules for the design of a perception/action network composed of one sensory module and two concurrent motor modules. Such a network is coupled with the dynamics of an inverted pendulum, showing spontaneous phase transitions toward various attractors, each attractor corresponding to a particular structural coupling within the agent/environment system. This spontaneous versatility is then exploited in a reinforcement learning paradigm where two separate reinforcement path are defined, one for the positive rewards, the other for the negative rewards. The learning experiment shows a fast adaptation to the constraints, followed by a slower phase where the behavior is improved. No degradation of the behavior is found for continuing learning, i.e. the learned behavior is preserved for long lasting time.