Stability of Coupled Local Minimizers Within the Lagrange Programming Network Framework

Coupled local minimizers (CLMs) turn out to be a potential global optimization strategy to explore a search space, avoid overfitting and produce good generalization. In this paper, convergence properties of CLMs based on an augmented Lagrangian function in the context of equality constrained minimization, are studied. We first consider the augmented Lagrangian by taking the objective of minimizing the average cost of an ensemble of local minimizers subject to pairwise synchronization constraints. Then we study an array of CLMs within the Lagrange programming network framework and analyze the local stability of CLMs using a linearization strategy. We further show that, under some mild conditions, global asymptotical stability of the unique equilibrium point of the network can be guaranteed. Afterwards, some sufficient conditions are presented to ensure the stability of synchronization between any two minimizers via a directed graph method. The results show that the CLMs usually can be synchronized if the penalty factors in the array of CLMs are chosen large enough. It is worth pointing out that CLMs possess the capability of global exploration in the search space and the advantage of faster running time on convex problems in comparison with most of the neural network approaches, which is also illustrated through two test functions and their numerical simulations.