Evolutionary Gradient Search Revisited

Evolutionary gradient search (EGS) is an approach to optimization that combines features of gradient strategies with ideas from evolutionary computation. Recently, several modifications to the algorithm have been proposed with the goal of improving its robustness in the presence of noise and its suitability for implementation on parallel computers. In this paper, the value of the proposed modifications is studied analytically. A scaling law is derived that describes the performance of the algorithm on the noisy sphere model and allows comparing it with competing strategies. The comparisons yield insights into the interplay of mutation, multire combination, and selection. Then, the covariance matrix adaptation mechanism originally formulated for evolution strategies is adapted for use with EGS in order to make the algorithm competitive on objective functions with large condition numbers of their Hessians. The resulting strategy is evaluated experimentally on a number of convex quadratic test functions.