Lambda and the Edge of Chaos in Recurrent Neural Networks

The idea that there is an edge of chaos, a region in the space of dynamical systems having special meaning for complex living entities, has a long history in artificial life. The significance of this region was first emphasized in cellular automata models when a single simple measure, λ_CA, identified it as a transitional region between order and chaos. Here we introduce a parameter λ_NN that is inspired by λ_CA but is defined for recurrent neural networks. We show through a series of systematic computational experiments that λ_NN generally orders the dynamical behaviors of randomly connected/weighted recurrent neural networks in the same way that λ_CA does for cellular automata. By extending this ordering to larger values of λ_NN than has typically been done with λ_CA and cellular automata, we find that a second edge-of-chaos region exists on the opposite side of the chaotic region. These basic results are found to hold under different assumptions about network connectivity, but vary substantially in their details. The results show that the basic concept underlying the lambda parameter can usefully be extended to other types of complex dynamical systems than just cellular automata.