Single- and multi-objective particle swarm optimization of reservoir structure in Echo State Network

Echo State Networks ESNs are specific kind of recurrent networks providing a black box modeling of dynamic non-linear problems. Their architecture is distinguished by a randomly recurrent hidden infra-structure called dynamic reservoir. Coming up with an efficient reservoir structure depends mainly on selecting the right parameters including the number of neurons and connectivity rate within it. Despite expertise and repeatedly tests, the optimal reservoir topology is hard to be determined in advance. Topology evolving can provide a potential way to define a suitable reservoir according to the problem to be modeled. This last can be mono- or multi-constrained. Throughout this paper, a mono-objective as well as a multi-objective particle swarm optimizations are applied to ESN to provide a set of optimal reservoir architectures. Both accuracy and complexity of the network are considered as objectives to be optimized during the evolution process. These approaches are tested on various benchmarks such as NARMA and Lorenz time series.