Simulation optimization with sensitivity information: An application to online-retail inventory replenishment

We study a simulation optimization approach for the online-retail inventory replenishment problem, where the goal is to minimize the total operational cost among a class of parametrized replenishment policies. We model the problem as an infinite-horizon average-cost dynamic program with discrete states and controls, which can be difficult to solve due to the curse of dimensionality. We propose a simulation optimization method that searches for low-cost policies using cost sensitivity information with respect to policy parameters. This information is analogous to the gradient in continuous-space descent methods, and we show that it can be collected as a by-product of the simulation process. Numerical experiments with realistic inventory networks show that our method is able to identify low-cost solutions efficiently, requiring significantly fewer iterations and less computation time than a simple random search method that only samples the local neighborhood.