Learning out of time series with an extended recurrent neural network

In this paper an extension to a regular recurrent neural network (ERNN) is presented. It allows to train the ERNN without the limitation of using input information just up to a preset future frame. It is possible to train the ERNN simultaneously in positive and negative time direction, leading in regression and classification experiments to results better than merging the outputs of separate networks trained in positive and negative time direction alone. The network structure is designed to be trained at least with any form of backpropagation through time. Structure and training procedure of the proposed network are explained. Results for classification experiments with an ERNN trained as a classifier and regression experiments with an ERNN trained to minimize the mean squared error on artificial data are reported and compared with previous approaches using merged outputs of regular RNNs. For real data, a classification experiment for speech feature vectors to phone classes is reported.