Predictive coding for dynamic vision: Development of functional hierarchy in a multiple spatio-temporal scales RNN model

The current paper presents a novel recurrent neural network model, predictive multiple spatio-temporal scales RNN (P-MSTRNN), which can generate as well as recognize dynamic visual patterns in a predictive coding framework. The model is characterized by multiple spatio-temporal scales imposed on neural unit dynamics through which an adequate spatio-temporal hierarchy develops via learning from exemplars. The model was evaluated by conducting an experiment of learning a set of whole body human movement patterns, which was generated by following a hierarchically defined movement syntax. The analysis of the trained model clarifies what types of spatio-temporal hierarchy develops in dynamic neural activity as well as how robust generation and recognition of movement patterns can be achieved by using the error minimization principle.