Non-Euclidean Monotone Operator Theory with Applications to Recurrent Neural Networks

We provide a novel transcription of monotone operator theory to the non-Euclidean finite-dimensional spaces ℓ1 and ℓ∞. We first establish properties of mappings which are monotone with respect to the non-Euclidean norms ℓ1 or ℓ∞. In analogy with their Euclidean counterparts, mappings which are monotone with respect to a non-Euclidean norm are amenable to numerous algorithms for computing their zeros. We demonstrate that several classic iterative methods for computing zeros of monotone operators are directly applicable in the non-Euclidean framework. We present a case-study in the equilibrium computation of recurrent neural networks and demonstrate that casting the computation as a suitable operator splitting problem improves convergence rates.