On-line optimization for a flow control strategy

We consider the problem of optimizing a threshold-dependent flow control strategy by using state information available on-line. It is assumed that a sample realization of a queueing system using this strategy is available, for which the threshold parameter is fixed. In contrast to earlier work, perturbation analysis in this case requires tracking queue lengths in addition to event times, in order to obtain performance sensitivity estimates. In this note we formally derive perturbation equations and describe their algorithmic implementation, which requires partial state memory in the form of arrival time information. Because this may impose serious storage limitations (unless the arrival process is deterministic), a capacity constraint is imposed. The effect of this constraint on our approach is finally investigated and experimental results are included.