Perceptual Quality Maximization for Video Calls With Packet Losses by Optimizing FEC, Frame Rate, and Quantization

We consider video calls affected by bursty packet losses, where frame-level forward error correction (FEC) is employed due to delay constraints, and damaged frames and others predicted from them are discarded. Here, a high frame rate (FR) at low bitrates leads to large quantization step sizes (QS), small frames, and suboptimal FEC, whereas a low FR at high bitrates reduces the perceptual quality. To mitigate frame losses and freezing, hierarchical-P (hierP) temporal layering can be used with lower coding efficiency than IPPP coding. We study the received quality maximization for hierP and IPPP, by jointly optimizing the encoding FR, QS, and the FEC redundancy, under sending bitrate constraints. Building upon Q-STAR perceptual quality and R-STAR bitrate models, which depend on QS, and the decoded and encoding FR, respectively, we cast the problem as a combinatorial optimization problem. We solve for the encoding FR and the bitrate using exhaustive search, hill-climbing, and a greedy FEC distribution algorithm to determine the FEC redundancies. We show that, for random losses, the FEC bitrate ratio is an affine function of the packet-loss rate; low encoding FR is preferred at wider sending bitrate ranges with higher packet loss; layer protection is more even at higher bitrates; and IPPP, while achieving higher Q-STAR scores, is prone to freezing. For bursty losses, we show that layer redundancies are higher, rising with the mean burst length, reaching 80%; and hierP achieves higher Q-STAR scores than IPPP for longer bursts, and a smaller mean and variance of decoded frame distances.