An Encoder–Decoder Model Based on Spiking Neural Networks for Address Event Representation Object Recognition

Address event representation (AER) object recognition task has attracted extensive attention in neuromorphic vision processing. The spike-based and event-driven computation inherent in the spiking neural network (SNN) provides an energy-saving solution for AER object recognition. However, SNN with spike timing dependent plasticity (STDP) learning rule has not achieved satisfying AER object recognition performance. This work proposes an SNN-based encoder-decoder model to improve the recognition performance of AER objects. An STDP-based locally connected spiking neural network (LC-SNN) is proposed as an encoder to extract rich spatiotemporal features from AER event flows more flexibly. After the encoder extracts and learns primary features, we propose a fully connected spiking neural network (FC-SNN) based on the reward-modulated spike-timing-dependent plasticity (R-STDP) learning rule as a decoder to learn higher-level features for classification. In addition, we improved the winner-take-all (WTA) mechanisms and R-STDP learning rule in the decoder based on the reward and punish decision, enabling the network to perform better. The experiments are performed on the N-MNIST, MNIST-DVS, and the dynamic vision sensor (DVS) gesture datasets, improving the accuracy of the best existing plasticity-based SNN by 0.19%, 0.27%, and 1.35%, respectively.