Discrete-time convergence theory and updating rules for neural networks with energy functions

We present convergence theorems for neural networks with arbitrary energy functions and discrete-time dynamics for both discrete and continuous neuronal input-output-functions. We discuss systematically how the neuronal updating rule should be extracted once an energy function is constructed for a given application, in order to guarantee the descent and minimization of the energy function as the network updates. We explain why the existing theory may lead to inaccurate results and oscillatory behaviors in the convergence process. We also point out the reason for and the side effects of using hysteresis neurons to suppress these oscillatory behaviors.