Rate-distortion optimized video streaming with adaptive playout

We propose a new scheme for streaming media systems that combines adaptive media playout (AMP) with rate-distortion (R-D)optimized packet transmission. AMP, the client-controlled, adaptive modification of the media playout rate, allows us to flexibly adjust the playout deadlines of individual packets, and can therefore reduce reconstruction distortion. This added flexibility incurs a subjective cost, however. We introduce functions that assess the subjective cost of a schedule of playout rate modifications, and we show how to optimize the schedule with respect to these costs and distortion. Because the optimal playout rate schedule and the R-D optimal transmission schedule are interdependent, we solve the two problems jointly. In simulations that model a receiver-driven scenario, results for a short media clip show a more than 2 dB improvement in mean PSNR for R-D optimized transmission scheduling combined with a moderate amount of AMP, over R-D optimized transmission scheduling alone.