Gradient Approximation and Multivariable Derivative-Free Optimization Based on Noncommutative Maps

In this article, multivariable derivative-free optimization algorithms for unconstrained optimization problems are developed. A novel procedure for approximating the gradient of multivariable objective functions based on noncommutative maps is introduced. The procedure is based on the construction of an exploration sequence to specify where the objective function is evaluated and the definition of so-called gradient generating functions which are composed with the objective function, such that the procedure mimics a gradient descent algorithm. Various theoretical properties of the proposed class of algorithms are investigated and numerical examples are presented.