Using Segmentation to Control the Retrieval of Data

One problem when storing sequential data using recurrent neural networks is that it is hard to preserve long term dependencies. Only the most recently stored data tend to be accurately recalled. One approach for reducing this recency effect has been to divide the data into segments and store the segments separately. This approach has provided promising results in prediction and classification domains. This paper analyzes in what way recall of the stored data is affected by segmentation. It is concluded that segmentation enables the control of which data that can be recalled. The problem of preserving long term dependencies in recurrent neural networks can therefore be reduced.