Evolving Generalized Modulatory Learning: Unifying Neuromodulation and Synaptic Plasticity

Neuromodulation and neuroplasticity work together to help organisms learn to cope in their environment. Experiments demonstrate that simple forms of neuromodulation aid in the learning process. In those studies, neuromodulation was used as a multiplier to scale the learning rate. However, more complex interactions have not been investigated. Our contributions are twofold: 1) we evolve a subnetwork that produces a modulatory signal that 2) is incorporated into the synaptic plasticity rule in nonlinear ways. In our experiments, we compute synaptic updates using a neural network and include the modulatory signal as one of the inputs. This allows evolution to combine the synaptic activity with modulation in highly nonlinear ways to arrive at weight updates. We show that organisms that evolve with this added complexity outperform simpler multiplicative neuromodulation, suggesting that gains might be won by investigating a broader class of interactions between neuromodulation and synaptic plasticity.