Embodied Neuromorphic Vision with Continuous Random Backpropagation

The brain outperforms computer architectures in aspects of energy efficiency, robustness and adaptivity. Brain computations are modeled in silico with spiking neural networks and neuromorphic hardware. Recently, three-factor synaptic plasticity rules approximating backpropagation have been derived. Suited to neuromorphic hardware, these rules can learn online with asynchronous updates. In this paper, we present Continuous Random Backpropagation (cRBP), a continuous version of Event-Driven Random Backpropagation. This learning rule performs comparably to state-of-the-art rules on the DvsGesture dataset. We additionally show that the accuracy can be significantly increased with a simple attention mechanism. This mechanism provides translation invariance at low computational cost compared to convolutions by exploiting event stream sparsity. Subsequently, we integrate cRBP in a real robotic setup, where a gripper grasps objects according to the detected visual affordances. In this setup, visual information is actively sensed by a Dynamic Vision Sensor (DVS) mounted on a robotic head performing microsaccadic eye movements. Our results suggest that advances in neuromorphic technology and plasticity rules enable the development of learning robots operating at high speed and low power.