Performance analysis of nonlinear echo state network readouts in signal processing tasks

Echo state networks (ESNs) characterize an attractive alternative to conventional recurrent neural network (RNN) approaches as they offer the possibility of preserving, to a certain extent, the processing capability of a recurrent architecture and, at the same time, of simplifying the training process. However, the original ESN architecture cannot fully explore the potential of the RNN, given that only the second-order statistics of the signals are effectively used. In order to overcome this constraint, distinct proposals promote the use of a nonlinear readout aiming to explore higher-order available information though still maintaining a closed-form solution in the least-squares sense. In this work, we review two proposals of nonlinear readouts - a Volterra filter structure and an extreme learning machine - and analyze the performance of these architectures in the context of two relevant signal processing tasks: supervised channel equalization and chaotic time series prediction. The obtained results reveal that the nonlinear readout can be decisive in the process of aproximating the desired signal. Additionally, we discuss the possibility of combining both ideas of nonlinear readouts and preliminary results indicate that a performance improvement can be attained.

I. Introduction

Recurrent neural networks (RNNs) are structures naturally adapted to deal with complex problems characterized by the existence of dynamical behavior, such as time series prediction, dynamical system identification and adaptive filtering [1]. This is a direct consequence of two factors: 1) the presence of feedback connections, which allow the development of an internal memory of the signal over time; and 2) the flexibility provided by nonlinear processing elements.