SQP-based Projection SPSA Algorithm for Stochastic Optimization with Inequality Constraints

Projection and penalty function simultaneous perturbation stochastic approximation (SPSA) algorithms are two commonly used methods in stochastic optimization problems under inequality constraints where no direct gradient of the loss function is available. However, both methods have potential shortcomings. We propose an algorithm that uses sequential quadratic programming (SQP) to estimate the projection operator under potentially complex explicit inequality constraints. This algorithm has some advantages over the penalty function method in practice. We prove the convergence of the proposed SQP-based projection SPSA algorithm and make a comparison with the penalty function method in a numerical example to show its superiority.