Flow control as stochastic optimal control problem with incomplete information

The nonlinear stochastic control problem related with flow control is considered. The state of the link is described by controlled hidden Markov process while the loss flow is described by the counting process with the intensity depending on the current transmission rate and unobserved link state. The control is the transmission rate and have to be chosen as non-anticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function taking into account the losses of transmitted information. Originally the problem belongs to a class of stochastic control with incomplete information, however, the optimal filtering equations giving the estimation of the current link state based on the observation of the loss process give the opportunity to reduce the problem to the standard stochastic control problem with full observations. Then, a necessary optimality condition is derived in the form of stochastic maximum principle which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. The optimal and suboptimal controls are investigated and compared with the flow control schemes which is used in TCP/IP networks. In particular, the optimal control demonstrates a much smoother behavior than the currently used TCP/IP congestion control.