Rollout algorithms: an overview

We review recent progress and open issues in the approximate solution of deterministic and stochastic optimization problems using rollout algorithms. These algorithms start with a heuristic policy and try to improve on that policy using on-line learning and simulation. They are related to dynamic programming and they are based on policy iteration ideas. Their attractive aspects are simplicity, broad applicability, and suitability for on-line implementation. While they do not aspire to optimal performance, rollout algorithms typically result in a consistent and substantial improvement over the underlying heuristic.