Quasi-Stochastic Approximation: Design Principles With Applications to Extremum Seeking Control

How can you optimize a function
$\Gamma{:}{\mathfrak{R}}^{d}\rightarrow{\mathfrak{R}}$
based on evaluations of this function without access to its gradient? Kiefer and Wolfowitz proposed a solution in the early 1950s based on stochastic approximation (SA)
[16]
, and in the 1920s, an engineer for the French railway system proposed an entirely deterministic approach that is now known as
extremum seeking control
(
ESC
)
[27]
,
[48]
. Once you understand the ESC architecture, you will find that the ideas are very similar. A fundamental difference is that random noise is replaced with sinusoids for exploration.