Time-series Forecasting Coding: A New Processing Method Developed from Predictive Coding for Recurrent Neural Networks

Recently, recurrent neural networks (RNNs) and the free energy principle (FEP) have been attracting much attention. FEP includes the so-called predictive coding (PC), in which the RNN input signals are coded as the difference between the RNN prediction output signals and original input signals. This paper proposes a new method for processing time-series data in RNNs, namely, time-series forecasting coding (TFC), developed by being based on the idea of PC. We consider multi-step ahead forecasting task with the following two schemes in our proposed TFC, i.e., smallest-delay feedback TFC and synchronous (temporally matching) feedback TFC. The latter is a natural extension of conventional PC. Experimental results show that the smallest-delay TFC presents the highest performance, while synchronous TFC shows comparable or lower performance, and then the conventional method indicates the lowest. This result suggests that the concept of PC is important even in time-series data processing, and that small delay is more important than synchronicity. This may have significant implications to the conceptual fundamentals of the PC and thus of the FEP.

I. Introduction

Artificial neural networks (ANNs) are a core information processing technology in the field of artificial intelligence and machine learning. They have been making remarkable progress in recent years, and are applied increasingly in the real world. One type of ANN is the recurrent neural networks (RNNs), having a recursive structure that makes it suitable for handling time-varying signals. They have been used in a wide range of applications, including industry, medicine, economics, and language [1].