SPSA in noise free optimization

The SPSA (simultaneous perturbation stochastic approximation) method for function minimization developed in Spall (1992) is analyzed for optimization problems without measurement noise. We prove the result that under appropriate technical conditions the estimator sequence converges to the optimum with geometric rate with probability 1. Numerical experiments support the conjecture that the top Lyapunov-exponent defined in terms of the SPSA method is smaller than the Lyapunov-exponent of its deterministic counterpart. We conclude that randomization improves convergence rate while dramatically reducing the number of function evaluations.