A note on neural networks with multiple equilibrium points

We give a condition which is necessary and sufficient for the injectivity (i.e., for the global invertibility) of vector fields defining a class of piece-wise-linear neural networks which include the Cellular Neural Networks as a special case. It is shown that this is the sharpest obtainable condition for injectivity, since it enables one to ascertain such property for each specific nonlinear piece-wise-linear function modeling the neuron activations. This result establishes an exact bound between neural circuits possessing a unique equilibrium point (which are tailor made, e.g., for solving global optimization problems) and those possessing multiple equilibrium points (which are suitable, e.g., for implementing a Content Addressable Memory or a Cellular Neural Network for image processing). We also prove conceptually similar results on injectivity in ease of continuously differentiable neuron activations. The proof of the main results exploits topological concepts from degree theory, such as the concept of homotopy of odd vector fields.